Services
  • Cloud Enablement Data & AI Digitalization End-to-End Digital Marketing SaaS
Industries
Products
  • Retail Productivity Healthcare Hospitality Cross-Industry App Development Testing & Monitoring
Company
Approach
Careers
Blog
Cloud Enablement Cloud Consulting Cloud-Native Apps Cloud Migration Strategies Cloud Migration Services Cloud Monitoring Cloud Security Posture AWS Cloud Cost Optimization Azure GCP App Engine Private Cloud
Data & AI Overview Data Science Data Engineering Big Data Processing Data Storage Data Management Data Visualization Machine Learning Video Analytics Artificial Intelligence Robotic Process Automation Computer Vision
Digitalization Mobility Extended Reality Web Development Internet of Things Blockchain Chatbots RTLS CRM eLearning Portals eCommerce Sites Intelligent Document Processing Product Information Management EAM
End-to-End Microservice Architecture UX / UI Design DevOps QA Automation Cybersecurity Performance Monitoring Frontend Monitoring API Management Compliance Consulting
Digital Marketing Overview Marketing Automation Analytics Paid Advertising SEO Social Media Content Marketing
SaaS Salesforce SharePoint Oracle HCM ServiceNow G Suite Microsoft Solutions Freshworks
Retail SlashQDigital Queue Management ContextIQRecommendation Engine PiQupDelivery Scheduling App
Productivity Notification AppNotification Builder KeverProject Management Tool QuickPicksSalesforce Widget
Healthcare Patient Transporter ManagementRTLS for Intra-Hospital Transfers
Hospitality TalQCall Accounting System
Cross-Industry SeeMyMachinesIIoT Solution
App Development Open Source WorksShared with the community
Testing & Monitoring PenQSecurity Testing Tool WebWatch24x7Server Monitoring Tool
Inquiry icon START A CONVERSATION

Share your requirements and we'll get back to you with how we can help.

Thank you for submitting your request.
We will get back to you shortly.

ETL Solution for Telecom Bill Processing

Client

US-based global telecom solutions expert that provides customer experience and call center technology to mid-sized and large businesses. The company sells cloud-based and on-premises software, contact center applications, and omnichannel customer engagement solutions.

Industry

Telecom and ISP

Overview

One of the major challenges involved in a data warehouse project is ensuring the quality of data. To address this issue, a data audit model was employed after weighing in custom business requirements during the Extract, Transform, Load (ETL) process. We architected the ETL process to develop telecom mediation layer data and generate billing, thereby providing a unified information base for the client’s bill processing system. Additionally, several issues in the existing system were fixed while testing the ETL process.

Show More Show Less

Business Requirement

The client wanted to architect, configure, and support their ETL process to develop telecom mediation layer data and generate invoices. This included dealing with large volumes of data across multiple source systems, transforming and loading data to and from data marts and data warehouses.

  • Parse file and derive multiple fields
  • Architect a scalable solution
  • Integrate with SureTax
  • Aggregate large volumes of data as per specifications

QBurst Solution

We used Apache Spark to design and implement the customized ETL process.

  • Python scripts pull telecom data as CSVs to Amazon S3
  • Spark application (daily aggregation) loads the CSVs and performs validations; validated data is used for derivation and enrichment; enriched data is stored in Parquet format
  • Another Spark application (monthly aggregation) performs rating, taxation, and saves billed data into database
  • SureTax integration helped to generate tax-included invoice

Benefits

  • Daily billing process reduced from 8 hours to 14 minutes (30 million records from over ​30,000 files)
  • Monthly run performed swiftly in 2.2 minutes
  • 14% increase in profit as a result of cost savings
  • 90% improvement in process efficiency
  • Improved access to billing information in a standard format
  • 25% faster retrieval and analysis of data

Technologies

  • Apache Spark
  • Amazon EMR
  • Amazon S3

Business Requirement

The client wanted to architect, configure, and support their ETL process to develop telecom mediation layer data and generate invoices. This included dealing with large volumes of data across multiple source systems, transforming and loading data to and from data marts and data warehouses.

  • Parse file and derive multiple fields
  • Architect a scalable solution
  • Integrate with SureTax
  • Aggregate large volumes of data as per specifications

QBurst Solution

We used Apache Spark to design and implement the customized ETL process.

  • Python scripts pull telecom data as CSVs to Amazon S3
  • Spark application (daily aggregation) loads the CSVs and performs validations; validated data is used for derivation and enrichment; enriched data is stored in Parquet format
  • Another Spark application (monthly aggregation) performs rating, taxation, and saves billed data into database
  • SureTax integration helped to generate tax-included invoice

Benefits

  • Daily billing process reduced from 8 hours to 14 minutes (30 million records from over ​30,000 files)
  • Monthly run performed swiftly in 2.2 minutes
  • 14% increase in profit as a result of cost savings
  • 90% improvement in process efficiency
  • Improved access to billing information in a standard format
  • 25% faster retrieval and analysis of data

Technologies

  • Apache Spark
  • Amazon EMR
  • Amazon S3
ETL Solution for Telecom

Download PDF

More Stories

More Stories

Online Auction Platform

View Case Study

Wind Farm IoT Solution

View Case Study

Business Process Integration

View Case Study