Behind the Streams: Real-Time Recommendations for Live Events Part 3

Key Actionable Insights

1

Split real-time update delivery into prefetch and broadcast phases to solve the thundering herd problem. Prefetch high-cardinality, compute-intensive data ahead of time by distributing requests naturally over a longer period, then trigger a low-cardinality broadcast at the critical moment to activate the pre-cached data on devices.

This approach is applicable whenever you need to update millions of clients simultaneously at a specific point in time, such as flash sales, live sports, or coordinated feature launches.

2

Add jitter to cache TTLs on both server and client sides to prevent synchronized cache expiration storms. Fixed TTLs cause all caches populated at similar times to expire simultaneously, creating unexpected traffic spikes that may occur hours before or after the actual peak event.

This pattern is especially important when user traffic is concentrated around specific times rather than evenly distributed, as synchronized cache refreshes can create mini thundering herds even outside peak windows.

3

Implement adaptive traffic prioritization with event-driven signals rather than relying solely on static traffic management rules. Route event-related traffic to dedicated clusters with more aggressive auto-scaling policies, and dynamically deprioritize non-critical server-driven updates during high RPS periods.

Be aware that deprioritizing traffic to non-critical services can cause unexpected call patterns and traffic spikes elsewhere in your system, requiring careful monitoring of downstream effects.

4

Design broadcast payloads to include both a state key and a timestamp to support at-least-once delivery semantics. The state key allows devices to look up pre-cached data locally without additional server requests, while the timestamp enables devices to catch up on missed broadcasts upon reconnection.

This is critical for unreliable network environments where devices may temporarily lose connectivity during live events, ensuring no viewer misses an update.

5

Use synthetic traffic generation to simulate game-day scenarios and identify potential traffic hotspots before live events. High-watermark traffic projections revealed issues like cache synchrony and unexpected traffic patterns that weren't visible in normal VOD traffic patterns.

Load testing with realistic burst patterns is essential because live event traffic behaves fundamentally differently from on-demand traffic, and issues may surface hours before or after the actual event window.

6

Keep business logic off client devices by using a map of stage keys in API responses that devices can look up locally. This allows the server to control the experience through broadcast state changes while devices simply render the appropriate pre-fetched content for each stage.

This pattern simplifies client-side logic and enables rapid iteration on the server side without requiring device updates, which is critical when supporting hundreds of millions of diverse devices.