Deploying More Reliable Apps with Uber Engineering’s XP Background Push

AJ Ribeiro
7 min readbeginner
--
View Original

Overview

The article discusses Uber Engineering's XP Background Push, a tool designed to enhance the reliability of mobile applications by enabling real-time bug fixes and feature management. It highlights the challenges of mobile app updates and how the XP Background Push addresses these issues through a push-based model, allowing for quicker responses to bugs without requiring users to manually update their apps.

What You'll Learn

1

How to implement XP Background Push for real-time bug fixes in mobile apps

2

Why push-based models are more effective than pull-based models for mobile app updates

3

When to use silent push notifications to manage feature flags in production

Prerequisites & Requirements

  • Understanding of mobile app architecture and feature flagging
  • Familiarity with push notification services like APNs and GCM(optional)

Key Questions Answered

How does XP Background Push improve mobile app reliability?
XP Background Push allows Uber to send silent push notifications that enable or disable feature flags in real-time, effectively mitigating bugs without requiring users to update their apps. This proactive approach ensures that users can continue using the app smoothly even when issues arise, enhancing overall reliability.
What challenges does Uber face with traditional mobile app updates?
Uber faces significant challenges with traditional mobile app updates, as once code is shipped, it cannot be retracted. Users must opt-in to updates, which can delay the resolution of bugs and outages, making it difficult to maintain app reliability at scale.
What is the role of the Treatment Service in XP Background Push?
The Treatment Service is responsible for evaluating feature flags and determining which features are enabled for users. It logs evaluation results, allowing Uber to manage feature rollouts and quickly disable problematic features when necessary.
How does Uber determine which users to notify about bug fixes?
Uber uses the Treatment Service to evaluate flag statuses for user/device pairs, enabling it to identify which users are affected by specific bugs. This targeted approach ensures that only those needing notifications receive them, optimizing resource use.

Key Statistics & Figures

Event processing rate
over 1,000 events per second
This rate reflects the performance of the pipeline that processes Treatment Service results for all flags across every Uber account.
Push notification sending rate
about 3,000 pushes per second
This rate indicates the efficiency of the GroupPusher service in delivering payloads to users.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Database
Cassandra
Used for storing the most recent Treatment Service results for all flags across Uber accounts.
Messaging
Kafka
Used for logging evaluation results of feature flags.
Stream Processing
Samza
Processes data from Kafka for database lookup.
Push Notification Service
Apns
Used for sending silent push notifications to iOS devices.
Push Notification Service
Gcm
Used for sending silent push notifications to Android devices.

Key Actionable Insights

1
Implement XP Background Push to enhance your mobile app's resilience against bugs.
By using silent push notifications to manage feature flags, you can ensure that users experience fewer disruptions and can continue using your app seamlessly, even when issues arise.
2
Utilize the Treatment Service to effectively manage feature rollouts and bug fixes.
This service allows you to dynamically control which features are active for users, enabling quick responses to issues without requiring app updates.
3
Adopt a push-based model for feature management to improve user experience.
This approach reduces the dependency on users to manually update their apps, allowing for faster resolutions to bugs and a more stable app environment.

Common Pitfalls

1
Failing to account for the timing of feature flag checks can lead to crashes.
If a feature flag is checked before the Treatment Service payload is received, it can result in a buggy feature being enabled, causing app crashes. To avoid this, ensure that feature flags are managed properly with a robust push notification system.

Related Concepts

Feature Flagging In Mobile Applications
Push Notification Architecture
Real-time Bug Management Strategies