ETL Solution for Telecom Bill Processing

Client

US-based global telecom solutions expert that provides customer experience and call center technology to mid-sized and large businesses. The company sells cloud-based and on-premises software, contact center applications, and omnichannel customer engagement solutions.

Industry

Telecom and ISP

Overview

One of the major challenges involved in a data warehouse project is ensuring the quality of data. To address this issue, a data audit model was employed after weighing in custom business requirements during the Extract, Transform, Load (ETL) process. We architected the ETL process to develop telecom mediation layer data and generate billing, thereby providing a unified information base for the client’s bill processing system. Additionally, several issues in the existing system were fixed while testing the ETL process.

Show More Show Less

Business Requirement

The client wanted to architect, configure, and support their ETL process to develop telecom mediation layer data and generate invoices. This included dealing with large volumes of data across multiple source systems, transforming and loading data to and from data marts and data warehouses.

Parse file and derive multiple fields
Architect a scalable solution
Integrate with SureTax
Aggregate large volumes of data as per specifications

QBurst Solution

We used Apache Spark to design and implement the customized ETL process.

Python scripts pull telecom data as CSVs to Amazon S3
Spark application (daily aggregation) loads the CSVs and performs validations; validated data is used for derivation and enrichment; enriched data is stored in Parquet format
Another Spark application (monthly aggregation) performs rating, taxation, and saves billed data into database
SureTax integration helped to generate tax-included invoice

Benefits

Daily billing process reduced from 8 hours to 14 minutes (30 million records from over 30,000 files)
Monthly run performed swiftly in 2.2 minutes
14% increase in profit as a result of cost savings
90% improvement in process efficiency
Improved access to billing information in a standard format
25% faster retrieval and analysis of data

Technologies

Apache Spark
Amazon EMR
Amazon S3

Business Requirement

Parse file and derive multiple fields
Architect a scalable solution
Integrate with SureTax
Aggregate large volumes of data as per specifications

QBurst Solution

We used Apache Spark to design and implement the customized ETL process.

Python scripts pull telecom data as CSVs to Amazon S3
Spark application (daily aggregation) loads the CSVs and performs validations; validated data is used for derivation and enrichment; enriched data is stored in Parquet format
Another Spark application (monthly aggregation) performs rating, taxation, and saves billed data into database
SureTax integration helped to generate tax-included invoice

Benefits

Daily billing process reduced from 8 hours to 14 minutes (30 million records from over 30,000 files)
Monthly run performed swiftly in 2.2 minutes
14% increase in profit as a result of cost savings
90% improvement in process efficiency
Improved access to billing information in a standard format
25% faster retrieval and analysis of data

Technologies

Apache Spark
Amazon EMR
Amazon S3

Download PDF

Cloud Enablement

Data & AI

Digitalization

End-to-End

SaaS

ETL Solution for Telecom Bill Processing

Client

Industry

Overview

Business Requirement

QBurst Solution

Benefits

Technologies

Business Requirement

QBurst Solution

Benefits

Technologies

More Stories

Online Auction Platform

Wind Farm IoT Solution

Business Process Integration