How we improved availability through iterative simplification

Solving and staying ahead of problems when scaling up a system of GitHub’s size is a delicate process. Here’s a look at some of the tools in GitHub’s toolbox, and how we’ve used them to solve problems.

Nick Hengeveld
7 min readintermediate
--
View Original

Overview

The article discusses how GitHub improved system availability through iterative simplification, focusing on the tools and methods used to address performance issues. It highlights specific improvements made to database queries and the removal of unused code, alongside key lessons learned during the process.

What You'll Learn

1

How to use Datadog for tracking metrics and identifying performance issues

2

Why incremental rollouts with Flipper can mitigate risks during deployments

3

How to leverage Scientist for testing performance impacts of code changes

Prerequisites & Requirements

  • Understanding of SQL query optimization techniques
  • Familiarity with Datadog and Splunk for monitoring and analysis(optional)

Key Questions Answered

How did GitHub improve the performance of SQL queries?
GitHub improved SQL query performance by changing the order of operations in their queries, specifically by querying user contributions first instead of repositories. This adjustment led to an 80-90% performance improvement, as confirmed by their Datadog experiment.
What tools does GitHub use to monitor system performance?
GitHub utilizes Datadog for tracking metrics related to events and traffic levels, and Splunk for analyzing context around issues. These tools help identify performance bottlenecks and improve overall system availability.
What lessons did GitHub learn from their performance optimization efforts?
GitHub learned that investing in observability is crucial for quickly identifying and solving problems. They also discovered the importance of looking for adjacent code issues and the need for gradual rollouts to measure the impact of changes.

Key Statistics & Figures

Performance improvement from query optimization
80-90%
This was achieved by changing the order of SQL queries related to user contributions.
Performance degradation from a previously tested SQL query
40-80%
This was observed when attempting to eliminate a SQL query and sort results in the application instead.
Final performance improvement from batching access checks
30-40%
This was confirmed through experiments that optimized how access checks were performed.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Monitoring
Datadog
Used for tracking metrics and identifying performance patterns.
Analytics
Splunk
Used for analyzing context around issues and troubleshooting.
Database
Mysql
Used for storing application data and managing queries.
Testing
Scientist
Used to test proposed code changes and measure their performance impact.
Feature Management
Flipper
Used to control the rollout of new features incrementally.

Key Actionable Insights

1
Implement a monitoring solution like Datadog to track key performance metrics in real-time.
This allows teams to identify performance issues proactively, rather than reactively addressing problems after they escalate.
2
Use feature flags with tools like Flipper to control the rollout of new features gradually.
This approach minimizes the risk of introducing bugs into production by allowing for controlled testing and rollback if necessary.
3
Conduct experiments with tools like Scientist to validate performance improvements before deploying changes.
This ensures that any modifications made to the codebase do not inadvertently degrade performance.

Common Pitfalls

1
Failing to monitor performance metrics can lead to undetected issues that escalate into critical problems.
Without proper observability, teams may only react to performance degradation after it has already affected users, making it harder to resolve.

Related Concepts

SQL Optimization
Performance Monitoring
Feature Flagging
Incremental Deployment Strategies