Business Challenges
Our client provides product information to leading retailers in the world acting as a middleman between suppliers/manufacturers and retailers. They use a proprietary framework to aggregate data from manufacturers and suppliers across the world. This data is transformed into a standard format that can be consumed by E-commerce websites and retailer inventory management systems.
- The current legacy framework could not keep pace with the increasing number of suppliers and custom data format requirements that flowed in from retailers and new-age E-commerce businesses.
- Requests for new data formats and custom integrations with manufacturers/suppliers were being handled as a new development with a time-to-production of more than eight weeks from analysis to development.
- Several transformation logic implementations like deduplication of product information based on fuzzy rules and tailoring content specific to retailers were time-consuming. The resulting volume of data was on the verge of going beyond the standard 24-hour service level agreement. They manage over 550k live product information in total.
- The customer wanted an alternate system that refined the aggregation process while reducing the development time from months to days. They wanted the solution to reduce dependency on their legacy ETL framework and its development teams by empowering data analysts with capabilities to perform data transformations and implementations.
QBurst Solution
The Talend-based unified data management platform significantly reduces development time and costs. The platform provides reusable jobs and custom routines that the data analysis team can easily use, even with minimal Talend expertise. Featuring a simple interface, the platform allows data analysts to implement business-critical transformations with minimal dependency on specialized integration teams. This helped them with the timely management of essential processes such as:
- Retail product information system updates
- Seamless management of master data
- Creation of large data sets
The solution leverages Talend Data Mapper as the central tool for orchestrating jobs and enabling efficient, scalable data transformations.