Overview
The article discusses the scalability and performance challenges faced by Espresso, LinkedIn's distributed document store, and outlines the solutions implemented to enhance its performance. Key improvements included migrating to HTTP/2, which resulted in significant reductions in latency and the number of TCP connections required.
What You'll Learn
1
How to implement HTTP/2 to improve scalability in distributed systems
2
Why connection multiplexing is essential for reducing latency
3
How to optimize request/response handling using Netty
Prerequisites & Requirements
- Understanding of distributed systems and network protocols
- Familiarity with Netty framework(optional)
Key Questions Answered
What are the performance improvements achieved by migrating to HTTP/2?
Migrating to HTTP/2 resulted in a 75% reduction in 99th and 99.9th percentile multi-read and read latencies, decreasing from 80ms to 20ms. Additionally, there was an 88% reduction in the number of TCP connections required between routers and storage nodes.
How does the new transport layer architecture enhance scalability?
The new architecture reduces the number of concurrent TCP connections between routers and storage nodes, addressing scalability challenges that arose from using HTTP/1.1. This transition allows for better resource management and improved system performance.
What optimizations were made to improve SSL performance?
The article describes offloading DNS lookups and handshakes to a separate thread pool, which prevents I/O threads from being blocked during SSL handshakes. Additionally, enabling Native SSL encryption/decryption with OpenSSL led to a 10% reduction in latency.
What changes were made to reduce garbage collection time?
The migration to HTTP/2 and the optimizations in request handling led to a 75% reduction in garbage collection times for both young and old generations. This was achieved by minimizing the creation of new pipelines and reusing existing channels.
Key Statistics & Figures
Latency reduction for single key get
7ms
from 20ms
Reduction in TCP connections
3.9 million
from 32 million
Garbage collection time for young generation
500ms
from 2000ms
Technologies & Tools
Backend
Netty
Used for the transport layer and HTTP/2 implementation.
Backend
Openssl
Utilized for Native SSL encryption/decryption to improve performance.
Key Actionable Insights
1Implementing HTTP/2 can significantly enhance the performance of distributed systems by enabling connection multiplexing.This is particularly useful in environments with high traffic where latency reduction is critical for user experience.
2Optimizing the request/response handling in Netty can lead to substantial performance improvements.By reusing channel pipelines, you can reduce memory usage and garbage collection, which is essential for maintaining high throughput in large-scale applications.
3Utilizing Native SSL can improve the efficiency of SSL operations in Java applications.Switching to a JNI-based SSL engine like OpenSSL can lead to reduced latency and better resource utilization.
Common Pitfalls
1
Failing to optimize the connection handling can lead to increased latency and resource consumption.
This often happens when developers do not consider the implications of connection pooling and multiplexing in high-load scenarios.
Related Concepts
Distributed Systems Architecture
Network Protocols
Performance Optimization Techniques