At the beginning of this year, we ran several experiments aimed at reducing the latency impact of the Ruby garbage collector (GC) in Shopify's monolith. In this article, Jean talks about the changes we made to improve GC performance, and more importantly, how we got to these changes.
Overview
This article discusses the efforts made by Shopify to improve the performance of the Ruby garbage collector (GC) in their monolithic application. It details the challenges faced, the strategies implemented, and the results achieved through various optimizations.
What You'll Learn
1
How to analyze garbage collection performance in Ruby applications
2
Why tuning garbage collector settings can impact application performance
3
How to implement out-of-band garbage collection in Ruby
Prerequisites & Requirements
- Understanding of Ruby's garbage collection mechanism
- Experience with performance optimization in web applications(optional)
Key Questions Answered
What changes were made to improve garbage collection performance in Shopify's monolith?
Shopify implemented several changes to their Ruby garbage collector settings, including tuning the oldmalloc limit and preallocating object slots. These adjustments aimed to reduce the frequency of major garbage collection events, which were identified as a primary cause of latency in request handling.
How does Ruby's garbage collector impact application latency?
Ruby's garbage collector pauses all execution when it runs, which can lead to increased latency for client requests. The article highlights that the median request spent 3ms in garbage collection, while the p99 request spent approximately 323ms, indicating significant performance bottlenecks during GC operations.
What is the significance of the 'shady' objects in Ruby's garbage collection?
'Shady' objects are those that cannot be promoted to the old generation due to missing write barriers. This leads to increased minor marking and can trigger major garbage collection events. The article explains how reducing the number of shady objects can enhance overall GC performance.
What results were observed after implementing out-of-band garbage collection?
After implementing out-of-band garbage collection, major marking during request cycles became infrequent, resulting in a significant reduction in tail latency. The article notes that the P99.99 and P99.9 latency times spent in GC were reduced almost tenfold, showcasing the effectiveness of this approach.
Key Statistics & Figures
Median request time spent in garbage collection
3ms
This is the median time for requests that logged GC information.
P99 request time spent in garbage collection
323ms
This indicates the latency experienced by the slowest 1% of requests.
Reduction in major marking events after tuning
20%
This was observed after adjusting the oldmalloc limit in production.
Reduction in shady objects from 160,000 to
3,000
This significant decrease was achieved through upstream contributions and optimizations.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Key Actionable Insights
1Regularly monitor garbage collection metrics to identify performance bottlenecks.By analyzing GC performance data, you can pinpoint areas for optimization and make informed decisions about tuning settings to improve application responsiveness.
2Consider implementing out-of-band garbage collection to minimize latency during request handling.This approach allows major GC events to occur during idle times, reducing the impact on user experience and improving overall application performance.
3Adjust the oldmalloc limit to better suit your application's memory allocation patterns.Setting a higher limit can reduce the frequency of major GC events, which is particularly beneficial for applications with high memory churn.
Common Pitfalls
1
Blindly applying garbage collection settings without proper measurement can lead to performance degradation.
It's crucial to establish a baseline and monitor the impact of any changes to ensure that they yield the desired improvements.
Related Concepts
Garbage Collection Mechanisms In Programming Languages
Performance Optimization Techniques
Memory Management Strategies