At GitHub, we serve tens of thousands of requests every second out of our network edge, operating on GitHub’s metal cloud. We’ve previously introduced GLB, our scalable load balancing solution…
Overview
GLB is GitHub's open source Layer 4 load balancer designed to efficiently distribute network traffic across multiple physical machines while minimizing connection disruptions. The article details its architecture, including the use of Equal-Cost Multi-Path (ECMP) routing and rendezvous hashing for consistent server selection.
What You'll Learn
1
How to implement a stateless load balancer using rendezvous hashing
2
Why using DPDK can enhance packet processing speeds in load balancers
3
How to manage server state transitions in a load balancing architecture
4
When to use Generic UDP Encapsulation (GUE) for packet transmission
Prerequisites & Requirements
- Understanding of load balancing concepts and network protocols
- Familiarity with DPDK and Linux networking tools(optional)
Key Questions Answered
What is GLB and how does it function as a load balancer?
GLB is GitHub's open source Layer 4 load balancer that distributes traffic across multiple servers using a single IP address. It minimizes connection disruptions during server changes by employing techniques like ECMP routing and rendezvous hashing to maintain consistent server selection for ongoing connections.
How does rendezvous hashing improve server selection in GLB?
Rendezvous hashing allows GLB to consistently select primary and secondary servers for incoming connections without maintaining state in the director tier. This ensures that even when servers are added or removed, existing connections can continue seamlessly, as the primary server can forward packets to the correct secondary server.
What role does DPDK play in GLB's performance?
DPDK enables GLB to achieve line rate packet processing by bypassing the Linux kernel, allowing it to handle high volumes of traffic efficiently. This is particularly crucial during DDoS attacks, where maintaining performance is essential to prevent bottlenecks in the load balancer.
How does GLB handle server failures gracefully?
GLB employs health checks to monitor backend servers and can swap primary and secondary entries in the forwarding table when a failure is detected. This soft drain approach allows existing connections to continue without disruption, enhancing reliability during server outages.
Key Statistics & Figures
Packet processing rate
Line rate processing on commodity NICs with commodity CPUs
This capability is crucial during DDoS attacks to prevent the load balancer from becoming a bottleneck.
Forwarding table size
65k rows
Each row contains a primary and secondary server IP address, allowing for efficient server selection.
Technologies & Tools
Packet Processing
Dpdk
Used for fast packet processing by bypassing the Linux kernel.
Load Balancing
Linux Virtual Server (lvs)
Provides stateful tracking for connections in traditional load balancing setups.
Packet Transmission
Generic UDP Encapsulation (gue)
Encapsulates packets to include secondary server metadata without router interference.
Key Actionable Insights
1Implementing rendezvous hashing in your load balancer can greatly enhance connection stability during server changes.By ensuring that the primary and secondary server selection remains consistent, you can avoid connection disruptions that typically occur when servers are added or removed from the pool.
2Utilizing DPDK for packet processing can significantly improve throughput and reduce latency in high-traffic environments.This is especially relevant for applications that require handling large volumes of requests, such as web services during peak traffic times.
3Regularly conducting health checks on backend servers is crucial for maintaining high availability in load balancing architectures.This proactive approach allows for quick responses to server failures, ensuring that user experience remains unaffected.
4Consider using Generic UDP Encapsulation (GUE) for efficient packet transmission within data centers.GUE allows for the inclusion of additional metadata in packets without causing routers to process them as unknown options, thus maintaining high throughput.
Common Pitfalls
1
Failing to maintain consistent server selection can lead to connection disruptions when servers are added or removed.
This often occurs in traditional load balancers that do not account for ongoing connections, leading to dropped packets and failed connections.
2
Over-reliance on stateless designs can result in challenges during server transitions.
While stateless designs simplify some aspects of load balancing, they can complicate connection management if not properly implemented.
Related Concepts
Load Balancing Techniques
Network Protocols And Routing
High Availability Architectures
Packet Processing Technologies