DNS Infrastructure at GitHub

Joe Williams

At GitHub we recently revamped how we do DNS from the ground up. This included both how we interact with external DNS providers and how we serve records internally to…

GitHub

•

Joe Williams

•8 min read•intermediate•

--

•View Original

MySQL

Overview

The article discusses GitHub's comprehensive overhaul of its DNS infrastructure to enhance scalability, performance, and visibility. It details the transition from a simple DNS configuration to a more robust, tiered architecture that supports both internal and external DNS zones across multiple data centers.

What You'll Learn

1

How to design a scalable DNS infrastructure for a large organization

2

Why separating DNS roles into caches, edges, and authorities improves performance

3

How to implement observability in DNS systems using metrics and logging

Prerequisites & Requirements

Understanding of DNS concepts and architecture
Familiarity with DNS server software like Unbound, NSD, and PowerDNS(optional)

Key Questions Answered

What were the main issues with GitHub's previous DNS infrastructure?

GitHub's previous DNS infrastructure faced performance and availability issues, causing degraded performance and outages. The system lacked metrics and visibility, making it difficult to identify problems, and it risked leaking internal network information due to its reliance on public DNS servers.

How does the new DNS architecture at GitHub improve scalability?

The new DNS architecture introduces a tiered design with separate roles for caches, edges, and authorities, allowing for better load distribution and reduced latency. Caches are located in each data center, minimizing cross-data center traffic and improving response times for DNS queries.

What technologies were chosen for GitHub's DNS infrastructure?

GitHub selected Unbound for caching, NSD for edge hosts, and PowerDNS for authorities. These technologies were chosen for their proven performance in larger DNS infrastructures and their ability to provide extensive metrics and logging.

How does GitHub ensure resilience in its DNS infrastructure?

GitHub's DNS infrastructure uses anycast IP addressing for caches, allowing requests to be routed to the nearest available cache. This design ensures low latency and provides fault tolerance, as requests can seamlessly switch to the next closest cache if one fails.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

DNS Caching

Unbound

Used for caching DNS responses to improve query performance.

DNS Authority

Nsd

Serves as the DNS authority for edge hosts.

DNS Authority

Powerdns

Acts as the authoritative DNS server for managing records.

Database

Mysql

Stores all DNS records for the infrastructure.

Key Actionable Insights

1
Implement a tiered DNS architecture to improve performance and scalability.
By separating the roles of caches, edges, and authorities, organizations can better manage DNS queries and reduce latency, which is crucial for high-traffic applications.

2
Utilize metrics and logging to enhance observability in DNS systems.
Having comprehensive metrics allows teams to quickly identify and troubleshoot issues, leading to improved uptime and performance.

3
Consider using anycast IP addressing for DNS caches.
Anycast enables efficient routing to the nearest cache, which can significantly reduce latency and improve resilience against server failures.

Common Pitfalls

1

Failing to monitor and log DNS queries can lead to undetected performance issues.

Without proper observability, teams may struggle to identify the root causes of DNS-related outages, leading to prolonged downtime and customer impact.

Related Concepts

DNS Architecture

Scalability In Distributed Systems

Observability In Infrastructure