Visit the post for more.
Overview
The article discusses how Facebook scales its use of memcached, a high-performance, distributed memory object caching system, to manage increasing demand. It details the modifications made to both the operating system and memcached to improve performance and efficiency.
What You'll Learn
1
How to implement a per-thread shared connection buffer pool for TCP and UDP sockets
2
Why using opportunistic polling can improve network I/O performance
3
How to reduce lock contention in multi-threaded applications
4
When to switch from TCP to UDP for network operations
5
How to batch dequeue packets to optimize network transmission
Key Questions Answered
How does Facebook scale memcached to handle increased demand?
Facebook scales memcached by implementing a per-thread shared connection buffer pool, using separate UDP sockets for replies, and optimizing network I/O through opportunistic polling. These changes have allowed them to handle 200,000 UDP requests per second with an average latency of 173 microseconds.
What modifications did Facebook make to improve memcached performance?
Facebook made several modifications, including implementing a shared connection buffer pool, using separate UDP sockets to reduce lock contention, and changing the dequeue algorithm to batch packets. These changes significantly improved memory efficiency and reduced latency.
What issues did Facebook encounter with UDP performance under load?
Under load, Facebook found that UDP performance suffered due to lock contention on the UDP socket lock. They addressed this by using separate UDP sockets for each thread, which improved performance without compromising backend efficiency.
What was the impact of using 8-core machines on memcached's performance?
With 8-core machines, Facebook discovered that the global lock for stats collection accounted for 20-30% of CPU usage. They eliminated this bottleneck by moving stats collection to a per-thread model, allowing for better performance.
Key Statistics & Figures
Memory reclaimed per server
multiple gigabytes
Achieved by implementing a per-thread shared connection buffer pool.
UDP requests handled per second
200,000
This performance was achieved after optimizations, with an average latency of 173 microseconds.
Throughput achieved
300,000 UDP requests/s
However, the latency at this request rate was deemed too high for practical use.
UDP requests/s using stock version
50,000
This was the performance level before Facebook's optimizations.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
Memcached
Used as a distributed memory object caching system to alleviate database load.
Operating System
Linux
The environment in which Facebook's memcached optimizations were implemented.
Key Actionable Insights
1Implement a per-thread shared connection buffer pool to optimize memory usage in high-traffic applications.This approach allows for significant memory reclamation, especially in environments with a large number of TCP connections, improving overall application performance.
2Utilize opportunistic polling for network interfaces to balance load across multiple cores.This technique can help alleviate bottlenecks caused by soft interrupts being handled by a single core, thus enhancing network I/O performance.
3Batch dequeue packets to minimize lock contention and improve throughput.By changing the dequeue algorithm to handle multiple packets at once, you can significantly reduce the overhead associated with lock acquisition in multi-threaded applications.
Common Pitfalls
1
Relying on a global lock for stats collection can severely impact performance in multi-core environments.
As seen in Facebook's case, this can account for a significant percentage of CPU usage, leading to inefficiencies. Moving to a per-thread model can alleviate this issue.
2
Not addressing lock contention in multi-threaded applications can lead to degraded performance.
The contention on the lock protecting the transmit queue can cause significant delays; optimizing the dequeue process can help mitigate this.