Surviving Flashes of High-Write Traffic Using Scriptable Load Balancers (Part I)

7 minute read This Sunday, over 100 million viewers will watch the Super Bowl. Whether they’re catching the match-up between the Falcons and the Patriots, or there for the commercials between the action, that’s a lot of eyeballs—and that’s only counting America. But all that attention doesn’t just stay on the screen, it gets directed to the web, and if you’re not prepared curious visitors could be rewarded with a sad error page. The Super Bowl makes us misty-eyed because our first big flash sale happened in 2007, after the Colts beat the Bears. Fans rushed online for T-shirts celebrating the win, giving us a taste of what can happen when a flood of people convene on one site in a very short duration of time. Since then, we’ve been continually levelling up our ability to handle flash sales, and our merchants have put us to the test: on any given day, they’ll hurl Super Bowl-sized traffic, often without notice.

Shopify logo
Shopify
•
Emil Stolarsky
•8 min read•intermediate•
--
•View Original
LuaMySQLRails

Overview

The article discusses how Shopify tackled the challenges of handling high-write traffic during flash sales using scriptable load balancers. It highlights the use of Nginx and Lua to implement a checkout throttle that manages traffic effectively while maintaining a positive customer experience.

What You'll Learn

1

How to use Nginx and Lua to manage high traffic loads

2

Why implementing a checkout throttle is essential for e-commerce platforms

3

How to apply the leaky bucket algorithm for traffic management

Prerequisites & Requirements

  • Basic understanding of load balancing and traffic management concepts
  • Familiarity with Nginx and Lua scripting(optional)

Key Questions Answered

How does Shopify handle high-traffic events like flash sales?
Shopify uses a combination of Nginx and Lua to create a checkout throttle that manages traffic during high-demand events. This system allows the platform to inform users of their queue status while preventing server overload, ensuring a smoother shopping experience.
What is the leaky bucket algorithm and how is it applied?
The leaky bucket algorithm is used to control the rate of requests to an application. In Shopify's case, it limits the number of requests processed in a given time frame, allowing only a set number of requests every 5 seconds, thus preventing overload during high-traffic sales.
What challenges did Shopify face during flash sales?
Shopify faced significant challenges with server overload during flash sales, particularly when high-traffic merchants like Kylie Cosmetics launched sales. These events could lead to site crashes and poor customer experiences, necessitating robust traffic management solutions.
How did customer experience impact Shopify's traffic management strategy?
Maintaining a positive customer experience was critical for Shopify. The platform needed to ensure that users were informed about their queue status without presenting error pages, which could lead to frustration and lost sales.

Key Statistics & Figures

Number of viewers during the Super Bowl
over 100 million
This statistic illustrates the potential traffic spikes that e-commerce platforms must prepare for during major events.
Time customers waited in queue
up to 40 minutes
This highlights the challenges faced by Shopify in managing user expectations during high-demand sales.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Backend
Nginx
Used for load balancing and managing high traffic loads.
Backend
Lua
Utilized within Nginx to create custom scripts for request handling.

Key Actionable Insights

1
Implement a checkout throttle using the leaky bucket algorithm to manage high traffic effectively.
This approach helps prevent server overload during peak times while maintaining a positive user experience by informing customers of their queue status.
2
Utilize Nginx and Lua for scriptable load balancing to enhance application performance.
This combination allows for efficient request handling and can significantly reduce the risk of downtime during high-traffic events.
3
Monitor customer sentiment on social media during high-traffic events to gauge user experience.
Real-time feedback can help identify issues quickly and inform necessary adjustments to traffic management strategies.

Common Pitfalls

1
Underestimating the impact of queue management on customer experience.
Shopify initially faced backlash from customers waiting too long in queues due to random polling, which could lead to frustration and lost sales. Prioritizing customers based on their entry time into the queue could improve satisfaction.

Related Concepts

Load Balancing
Traffic Management
E-commerce Performance Optimization