Visit the post for more.
Overview
The article discusses Facebook's transition from IPv4 to IPv6 in its data center infrastructure, focusing on how to maintain legacy IPv4 support while operating an IPv6-only environment. It outlines the use of load balancers to handle IPv4 traffic and the technical adaptations made to support this transition.
What You'll Learn
1
How to implement IPv4 support in an IPv6-only infrastructure using load balancers
2
Why using an IPv4 link-local address as a BGP next-hop is beneficial in IPv6-only environments
3
How to create a custom kernel module for decapsulating mixed-version IP tunnels
Key Questions Answered
How does Facebook maintain IPv4 support in an IPv6-only infrastructure?
Facebook maintains IPv4 support by using load balancers that handle IPv4 traffic while the internal infrastructure remains IPv6-only. The L4 load balancer wraps IPv4 requests in IPv6 tunnels, allowing seamless communication without requiring routable IPv4 addresses on servers.
What changes were made to load balancers to support IPv4 traffic?
The L4 load balancer was modified to tunnel IPv4 traffic inside IPv6, allowing it to forward requests to the L7 load balancer. This setup avoids the need for routable IPv4 addresses on the servers while still enabling them to respond to IPv4 clients.
What is the role of the L7 load balancer in Facebook's infrastructure?
The L7 load balancer receives IPv4 requests encapsulated in IPv6 tunnels, decapsulates them, and processes the requests as usual. It uses a direct server return design to send responses directly to clients, bypassing the L4 load balancer.
What is the significance of using a custom kernel module for tunneling?
The custom kernel module allows for decapsulation of both same-version and mixed-version IP tunnels, which is crucial for handling requests in an IPv6-only environment where traditional tunneling methods are insufficient.
Key Statistics & Figures
Percentage of internal traffic that is IPv6
99 percent
This statistic highlights the extent of Facebook's transition to IPv6.
Percentage of Facebook users with IPv6 support
15 percent
This indicates the challenge of serving the majority of users who still rely on IPv4.
Percentage of data center clusters that are IPv6-only
50 percent
This shows the progress made in transitioning data center infrastructure to IPv6.
Technologies & Tools
Networking
Bgp
Used for routing traffic within the Facebook network.
Networking
Exabgp
Used to announce virtual IP addresses for load balancers.
Networking
Ipvs
Used by the L4 load balancer to forward traffic to the L7 load balancer.
Networking
Netlink
Used in the development of gnlpy for setting up IPVS forwarding.
Operating System
Linux
The platform on which the custom kernel module for decapsulation was developed.
Software
Gnlpy
An open-sourced library created to facilitate IPVS forwarding setup.
Key Actionable Insights
1Implementing IPv4 support in an IPv6-only environment can be achieved by modifying load balancers to handle encapsulation and decapsulation of traffic.This approach allows organizations to transition to IPv6 while still serving clients that rely on IPv4, ensuring a smoother migration process.
2Using link-local addresses as BGP next-hops simplifies routing in an IPv6-only infrastructure.This method avoids the complexity of managing routable IPv4 addresses, allowing for efficient traffic flow between load balancers and routers.
3Creating custom kernel modules can enhance functionality in specialized networking scenarios.By developing a module to handle mixed-version tunnels, organizations can address specific needs that standard solutions do not cover.
Common Pitfalls
1
Relying solely on standard tunneling methods can lead to limitations in mixed-version IP environments.
Organizations may find that default Linux tunneling modules do not support the necessary functionality, necessitating custom solutions.