Inquiry icon

START A CONVERSATION

Share your requirements and we'll get back to you with how we can help.

Thank you for submitting your request.
We will get back to you shortly.

ETL Solution for Telecom Bill Processing

Client

US-based global telecom solutions expert that provides customer experience and call center technology to mid-sized and large businesses. The company sells cloud-based and on-premises software, contact center applications, and omnichannel customer engagement solutions.

Industry

Telecom and ISP

Overview

One of the major challenges involved in a data warehouse project is ensuring the quality of data. To address this issue, a data audit model was employed after weighing in custom business requirements during the Extract, Transform, Load (ETL) process. We architected the ETL process to develop telecom mediation layer data and generate billing, thereby providing a unified information base for the client’s bill processing system. Additionally, several issues in the existing system were fixed while testing the ETL process.

Show More Show Less

Business Requirement

The client wanted to architect, configure, and support their ETL process to develop telecom mediation layer data and generate invoices. This included dealing with large volumes of data across multiple source systems, transforming and loading data to and from data marts and data warehouses.

  • Parse file and derive multiple fields
  • Architect a scalable solution
  • Integrate with SureTax
  • Aggregate large volumes of data as per specifications

QBurst Solution

We used Apache Spark to design and implement the customized ETL process.

  • Python scripts pull telecom data as CSVs to Amazon S3
  • Spark application (daily aggregation) loads the CSVs and performs validations; validated data is used for derivation and enrichment; enriched data is stored in Parquet format
  • Another Spark application (monthly aggregation) performs rating, taxation, and saves billed data into database
  • SureTax integration helped to generate tax-included invoice

Benefits

  • Daily billing process reduced from 8 hours to 14 minutes (30 million records from over ​30,000 files)
  • Monthly run performed swiftly in 2.2 minutes
  • 14% increase in profit as a result of cost savings
  • 90% improvement in process efficiency
  • Improved access to billing information in a standard format
  • 25% faster retrieval and analysis of data

Technologies

  • Apache Spark
  • Amazon EMR
  • Amazon S3

Business Requirement

The client wanted to architect, configure, and support their ETL process to develop telecom mediation layer data and generate invoices. This included dealing with large volumes of data across multiple source systems, transforming and loading data to and from data marts and data warehouses.

  • Parse file and derive multiple fields
  • Architect a scalable solution
  • Integrate with SureTax
  • Aggregate large volumes of data as per specifications

QBurst Solution

We used Apache Spark to design and implement the customized ETL process.

  • Python scripts pull telecom data as CSVs to Amazon S3
  • Spark application (daily aggregation) loads the CSVs and performs validations; validated data is used for derivation and enrichment; enriched data is stored in Parquet format
  • Another Spark application (monthly aggregation) performs rating, taxation, and saves billed data into database
  • SureTax integration helped to generate tax-included invoice

Benefits

  • Daily billing process reduced from 8 hours to 14 minutes (30 million records from over ​30,000 files)
  • Monthly run performed swiftly in 2.2 minutes
  • 14% increase in profit as a result of cost savings
  • 90% improvement in process efficiency
  • Improved access to billing information in a standard format
  • 25% faster retrieval and analysis of data

Technologies

  • Apache Spark
  • Amazon EMR
  • Amazon S3
ETL Solution for Telecom

Download PDF

More Stories

Online Auction Platform

View Case Study >

Wind Farm IoT Solution

View Case Study >

Business Process Integration

View Case Study >