Kubernetes at GitHub

Jesse Newland

Over the last year, GitHub has gradually evolved the infrastructure that runs the Ruby on Rails application responsible for github.com and api.github.com. We reached a big milestone recently: all web…

GitHub

•

Jesse Newland

•14 min read•intermediate•

--

•View Original

AWSGolangKubernetesPuppetRailsRubyTerraformYAML

Overview

The article discusses GitHub's transition of its Ruby on Rails application infrastructure to Kubernetes, detailing the challenges faced and the lessons learned throughout the migration process. It highlights the benefits of Kubernetes in providing a self-service platform for deploying and scaling applications efficiently.

What You'll Learn

1

How to deploy applications to Kubernetes clusters effectively

2

Why migrating to Kubernetes can improve application scalability and reliability

3

When to use self-service deployment tools in a large application environment

Prerequisites & Requirements

Understanding of container orchestration concepts
Familiarity with Kubernetes and its components(optional)

Key Questions Answered

What were the main reasons for GitHub's migration to Kubernetes?

GitHub migrated to Kubernetes to address scalability issues, allow teams to deploy independently, and reduce server maintenance time. The previous infrastructure was outdated and could not support the increasing demand for new features and services, leading to a need for a more flexible and self-service platform.

How did GitHub ensure confidence in their Kubernetes deployment?

GitHub built confidence in their Kubernetes deployment by conducting extensive testing, including simulating production traffic and performing failure tests. They gradually increased the load on the Kubernetes cluster while monitoring performance and reliability, allowing internal users to opt-in to the new infrastructure for real-world feedback.

What challenges did GitHub face during the migration to Kubernetes?

During the migration, GitHub faced challenges such as ensuring low-latency access to data services, managing kernel panics on nodes, and maintaining service availability during the transition. These challenges required careful planning and testing to ensure a smooth migration.

Key Statistics & Figures

Deployments per day

dozens of times

This indicates the frequency of code deployments to Kubernetes clusters after migration.

Percentage of web traffic served by Kubernetes

100%

This reflects the complete transition of GitHub's frontend services to Kubernetes after the migration.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Container Orchestration

Kubernetes

Used to automate deployment, scaling, and management of containerized applications.

Infrastructure As Code

Terraform

Used for managing the Kubernetes cluster in an AWS VPC.

Networking

Calico

Selected as the network provider for Kubernetes clusters.

Key Actionable Insights

1
Implement a self-service deployment platform to empower engineering teams.
By allowing teams to deploy their own services, GitHub reduced the burden on Site Reliability Engineers and improved deployment speed. This approach can be applied in any organization looking to enhance agility and reduce operational overhead.

2
Conduct thorough testing and simulations before a full migration.
GitHub's approach of gradually increasing traffic to the Kubernetes cluster helped identify potential issues early. Organizations should adopt similar testing strategies to ensure reliability and performance before making significant infrastructure changes.

Common Pitfalls

1

Underestimating the complexity of migrating critical workloads to Kubernetes.

Many organizations may overlook the necessary planning and testing required for a successful migration. It's crucial to conduct thorough evaluations and simulations to identify potential issues before full deployment.

Related Concepts

Container Orchestration

Self-service Deployment

Microservices Architecture