Overview
The article discusses Shopify's transition to a more scalable ingress solution using ingress-nginx, highlighting the challenges faced with the previous Google Cloud Load Balancer Controller and the optimizations made to improve performance. It details the implementation of dynamic configuration to reduce NGINX reloads during deployments, enhancing request latencies significantly.
What You'll Learn
1
How to implement dynamic configuration in NGINX to reduce reloads
2
Why endpoint awareness is crucial for load balancing in Kubernetes
3
When to transition from Google Cloud Load Balancer to ingress-nginx
Prerequisites & Requirements
- Understanding of Kubernetes and NGINX
- Familiarity with Lua and its integration with NGINX(optional)
Key Questions Answered
What were the performance issues with Google Cloud Load Balancer?
Shopify experienced underperforming load balancing and request queueing with Google Cloud Load Balancer, particularly during deployments. This led to increased latencies for end users, prompting the transition to ingress-nginx, which offered endpoint awareness and better performance.
How does dynamic configuration improve NGINX performance?
Dynamic configuration allows ingress-nginx to update application endpoints in-memory, eliminating the need for NGINX to reload its configuration file during deployments. This significantly reduces request latencies and improves overall performance, especially during high-load scenarios.
What were the results of the latency tests between glbc and ingress-nginx?
The tests showed that while both ingress solutions performed similarly up to the 99.9th percentile of request latencies, ingress-nginx outperformed glbc significantly at the 99.99th percentile and beyond, highlighting its efficiency under load.
What is the impact of NGINX reloads on performance?
NGINX reloads, while graceful, increase memory consumption and reset keepalive connections, which can lead to higher latencies during deployments. The introduction of dynamic configuration mitigates these issues by maintaining endpoint configurations in-memory.
Key Statistics & Figures
Number of merchants on Shopify platform
600,000+
This growth from 375,000 since 2016 indicates the scale at which Shopify operates.
Daily deployments at Shopify
40 times
This high frequency of deployments necessitates efficient load balancing and ingress management.
Latency improvement at 99.99th percentile
Multiple orders of magnitude better
Ingress-nginx outperformed glbc significantly at high latency percentiles, crucial for merchant success.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Orchestration
Kubernetes
Used to manage and scale Shopify's cloud infrastructure.
Web Server
Nginx
Serves as the ingress controller for routing incoming requests.
Scripting
Lua
Used in conjunction with NGINX to handle dynamic configuration updates.
Key Actionable Insights
1Implement dynamic configuration in your NGINX setup to minimize reloads during deployments.This approach not only reduces latency but also enhances user experience during application updates, making it essential for high-traffic environments.
2Consider transitioning to ingress-nginx if you are currently using Google Cloud Load Balancer and facing performance issues.The enhanced endpoint awareness and load balancing capabilities of ingress-nginx can significantly improve application performance and reliability.
3Regularly test and monitor latency metrics during deployments to identify performance bottlenecks.Understanding how different ingress solutions perform under load will help you make informed decisions about infrastructure optimizations.
Common Pitfalls
1
Failing to account for the performance impact of NGINX reloads during deployments.
Reloads can lead to increased latencies and resource consumption, which can degrade user experience. Implementing dynamic configuration can help mitigate these issues.
Related Concepts
Kubernetes Ingress Management
Load Balancing Strategies
Performance Optimization Techniques