Overview
The article discusses Uber's 'Orders Near You' feature, which utilizes real-time geospatial data analytics to enhance user experience in the Uber Eats app. It highlights the technical implementation using Apache Pinot for low-latency analytics and the architectural insights gained from separating OLTP and OLAP systems.
What You'll Learn
1
How to implement real-time analytics using Apache Pinot
2
Why geospatial indexing is crucial for high-QPS applications
3
When to separate OLTP and OLAP systems for better performance
Prerequisites & Requirements
- Understanding of geospatial data concepts
- Familiarity with Apache Pinot(optional)
Key Questions Answered
How does Uber utilize real-time geospatial data in their app?
Uber uses real-time geospatial data to provide insights into what nearby users are ordering through the 'Orders Near You' feature. This feature is designed to enhance user engagement by showcasing popular food choices in real-time, thereby encouraging discovery and increasing orders.
What are the performance benefits of using Apache Pinot?
Apache Pinot allows Uber to achieve query latencies of under 50ms for the 'Orders Near You' feature, significantly improving performance compared to previous architectures that required multiple service calls. This efficiency is crucial for handling thousands of queries per second while maintaining low latency.
What challenges did Uber face with their initial architecture?
The initial architecture faced scalability challenges, requiring up to 20 calls to the order-gateway service per user request, leading to hundreds of thousands of Cassandra lookups per second. This resulted in capacity shortages and necessitated a redesign to improve efficiency.
What insights were gained from separating OLTP and OLAP systems?
Separating OLTP and OLAP systems allowed Uber to optimize for different query patterns, improving reliability and performance. This architectural change ensures that analytical loads do not impact transactional processing, thereby enhancing overall system efficiency.
Key Statistics & Figures
Query latency
under 50ms
This performance metric is critical for the 'Orders Near You' feature, enabling it to handle thousands of queries per second.
Cassandra capacity increase
6x
The initial architecture required this increase to manage the heavy analytical read traffic generated by the application.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
Apache Pinot
Used for real-time analytics and low-latency querying of geospatial data.
Database
Apache Cassandra
Initially used as a permanent store for placed Eats orders.
Messaging
Apache Kafka
Used to publish modification logs from OLTP databases to the OLAP system.
Stream Processing
Apache Flink
Optionally transforms Kafka events before ingestion into Pinot.
Key Actionable Insights
1Implement geospatial indexing in your applications to enhance query performance.Geospatial indexing allows for faster retrieval of location-based data, which is essential for applications requiring real-time analytics, such as delivery services.
2Consider separating OLTP and OLAP systems to improve reliability and performance.This separation helps isolate analytical workloads from transactional processing, reducing the risk of performance degradation in operational databases.
3Utilize Apache Pinot for low-latency analytics on large datasets.Pinot's architecture is designed for high query throughput and low latency, making it suitable for applications that require real-time insights from massive data streams.
Common Pitfalls
1
Overloading a single database with both OLTP and OLAP workloads can lead to performance issues.
This often results in slow response times and increased latency, as seen in Uber's initial architecture where Cassandra struggled to handle high read traffic.
2
Neglecting to implement geospatial indexing can hinder query performance.
Without proper indexing, applications may face scalability challenges, especially when processing location-based queries in real-time.
Related Concepts
Geospatial Data Analytics
Real-time Data Processing
Oltp Vs Olap Architectures