Unified IT Observability Solution

Headquartered in Germany, our client is the research and development center for the world’s largest manufacturer of premium and commercial vehicles. The center focuses on research, IT engineering, and product development.

• Fragmented Data: Telemetry data was scattered across disparate systems, making it difficult to gain a holistic view of application performance.
• Lack of Standardization: Without a unified approach, each project generated observability data differently, complicating analysis and troubleshooting for operations teams.
• High Developer Overhead: Developers spent excessive time on manual tasks to instrument and manage observability, diverting their focus from core product development.
• Vendor Lock-in Risk: Reliance on proprietary solutions would create a dependency on a single vendor and incur high licensing costs.

QBurst designed and implemented a unified observability platform using well-established open-source technologies. The solution uses OpenTelemetry for standardized data generation and collection and OpenSearch for centralized storage and visualization. We customized these tools to operate within the client's secure, restricted container environment.

Key components:

• OpenTelemetry for Data Collection: We implemented and extended the OpenTelemetry log model to capture company-specific attributes, ensuring a standardized approach to data collection across all projects.
• OpenSearch for Storage & Visualization: The platform uses OpenSearch as a unified data sink for logs, traces, and metrics. OpenSearch Dashboards provides a centralized interface for visualizing this data, enabling teams to quickly analyze performance and identify issues.
• Automated Deployment: We developed a simple installer using Helm to automate the deployment of the entire platform, ensuring it is ready for immediate use.
• Reusable Assets: We created a library of reusable OpenSearch objects, including dashboards and machine learning models for anomaly detection, which can be shared and leveraged by various teams.

• End-to-End Encryption: The entire platform architecture is end-to-end encrypted to ensure the security of all telemetry data.
• Kubernetes Integration: Used Kubernetes annotations for automatic signal generation and propagation, and Kubernetes Cron Jobs for automated index maintenance (compression, archival, deletion).
• Automated Policies: Implemented policies for automatic scaling and load balancing, ensuring the platform can handle increasing data volumes.
• OIDC Compatibility: Ensured seamless compatibility with the client's existing OpenID Connect (OIDC) authentication and authorization toolchain.

• Faster Issue Resolution: Developers now have a single, comprehensive view of application performance, allowing them to identify and resolve issues quickly.
• Increased Developer Productivity: The automated, plug-and-play nature of the platform reduced the workload on developers, allowing them to focus on core product innovation.
• Improved Operational Efficiency: Standardized observability models enabled operations teams to manage a larger number of projects more effectively with a comprehensive, consistent view.
• Vendor Freedom: The open-source foundation of the solution ensures there is no vendor lock-in, providing the flexibility to easily switch to other data sinks in the future.