Solving and staying ahead of problems when scaling up a system of GitHub’s size is a delicate process. Here’s a look at some of the tools in GitHub’s toolbox, and how we’ve used them to solve problems.
Overview
The article discusses how GitHub improved system availability through iterative simplification, focusing on the tools and methods used to address performance issues. It highlights specific improvements made to database queries and the removal of unused code, alongside key lessons learned during the process.
What You'll Learn
How to use Datadog for tracking metrics and identifying performance issues
Why incremental rollouts with Flipper can mitigate risks during deployments
How to leverage Scientist for testing performance impacts of code changes
Prerequisites & Requirements
- Understanding of SQL query optimization techniques
- Familiarity with Datadog and Splunk for monitoring and analysis(optional)
Key Questions Answered
How did GitHub improve the performance of SQL queries?
What tools does GitHub use to monitor system performance?
What lessons did GitHub learn from their performance optimization efforts?
Key Statistics & Figures
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Key Actionable Insights
1Implement a monitoring solution like Datadog to track key performance metrics in real-time.This allows teams to identify performance issues proactively, rather than reactively addressing problems after they escalate.
2Use feature flags with tools like Flipper to control the rollout of new features gradually.This approach minimizes the risk of introducing bugs into production by allowing for controlled testing and rollback if necessary.
3Conduct experiments with tools like Scientist to validate performance improvements before deploying changes.This ensures that any modifications made to the codebase do not inadvertently degrade performance.