Safety Check: Streamlining deployment around the world

Visit the post for more.

Peter Cottle
10 min readintermediate
--
View Original

Overview

The article discusses Facebook's Safety Check feature, which allows users to notify friends and family of their safety during crises. It outlines the technical challenges faced in deploying this feature globally, including the algorithm used for activation and the infrastructure improvements made to streamline the process.

What You'll Learn

1

How to implement a push notification system for crisis management

2

Why leveraging social graphs can enhance user engagement during emergencies

3

How to automate deployment processes for emergency response features

Key Questions Answered

How does Facebook's Safety Check identify users in affected areas?
Safety Check utilizes a push notification model that prompts users in crisis areas to mark themselves safe. It leverages the social graph to identify friends in the affected area, which allows for rapid dissemination of safety statuses through notifications.
What challenges did Facebook face in scaling Safety Check activations?
Facebook faced challenges related to efficiently identifying users in affected areas, managing job scheduling to avoid overload, and ensuring system stability during high-demand situations. They implemented throttling and rate-limiting to manage the load effectively.
What is the purpose of the internal rate-limiting service mentioned in the article?
The internal rate-limiting service is designed to manage the number of jobs scheduled during Safety Check activations. It helps maintain a specified queries per second (QPS) limit, preventing system overload and ensuring smooth operation during crisis notifications.
How does Facebook ensure Safety Check is ready for immediate deployment?
Facebook conducts proactive testing through 'shadow launches' every 12 hours, which simulate the activation process without sending notifications. This allows them to verify the system's readiness and performance across various conditions.

Key Statistics & Figures

Number of Safety Check activations
17 times
Between January and May 2016, compared to 11 times in 2014 and 2015 combined.
Number of people notified of safety statuses
more than a billion
Since the first activation in December 2014 for Typhoon Ruby.
Graph traversal efficiency
exhaust a small graph of ~100,000 people in just minutes
This demonstrates the speed of the algorithm in reaching users during activations.

Key Actionable Insights

1
Implement a proactive testing system for critical features to ensure readiness during emergencies.
By conducting regular shadow launches, teams can identify potential issues and ensure that systems like Safety Check are functional when needed most.
2
Utilize social graphs to enhance user engagement and response during crises.
Leveraging the relationships between users can significantly improve the speed and effectiveness of crisis communication, as seen with Safety Check's algorithm.
3
Incorporate automated deployment tools to reduce reliance on specific team members during emergencies.
This allows for a more flexible and responsive approach to activating features like Safety Check, ensuring that help can be mobilized at any time.

Common Pitfalls

1
Over-scheduling jobs during initial activation due to unsynchronized counters.
This can lead to system overload and requires manual intervention. To avoid this, implement a delay in job scheduling during the initial seconds of activation.