Overview
The article discusses Uber's implementation of an automated vertical CPU scaling system that optimizes resource allocation for storage workloads, leading to significant cost savings and improved reliability. It details the methodology behind determining the appropriate CPU core allocation for over 500,000 Docker containers, resulting in a reduction of more than 120,000 cores and millions in annual infrastructure savings.
What You'll Learn
1
How to implement automated vertical CPU scaling for containerized workloads
2
Why monitoring CPU utilization is crucial for resource allocation
3
When to apply peak CPU utilization metrics for scaling decisions
Prerequisites & Requirements
- Understanding of CPU utilization metrics and container orchestration
- Familiarity with monitoring tools like M3 and cexporter(optional)
Key Questions Answered
How does Uber's vertical CPU scaling system work?
Uber's vertical CPU scaling system automates the allocation of CPU cores to storage workloads based on historical CPU utilization data. By analyzing the peak CPU usage over a two-week period, the system determines the optimal core count for each container, ensuring efficient resource use and minimizing costs.
What are the benefits of using CPU utilization metrics for scaling?
Using CPU utilization metrics allows for a standardized approach to resource allocation across various storage technologies. This method ensures consistent performance, reduces the risk of overprovisioning, and simplifies the scaling process by relying on readily available data instead of manual adjustments.
What challenges are associated with determining the right CPU core count for containers?
Challenges include the variability of storage workloads, which can change significantly throughout the week, and the need for a model that accommodates different use cases. Additionally, frequent movements of workloads between hosts can complicate core allocation decisions.
How much has Uber saved through vertical CPU scaling?
Since implementing vertical CPU scaling, Uber has achieved a reduction of over 120,000 cores, translating into annual savings of millions of dollars in infrastructure costs while improving the reliability of their platform.
Key Statistics & Figures
Reduction in CPU core allocations
120,000 cores
This reduction has led to multi-million dollar savings in infrastructure spending.
Number of Docker containers managed
500,000 containers
The automated vertical CPU scaling system is currently applied to these containers.
Total CPU cores in use
2.5 million CPU cores
Uber operates over 1,000,000 storage containers across nearly 75,000 hosts.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Containerization
Docker
Used to run storage workloads in a containerized environment.
Monitoring
M3
Monitoring stack used to collect and analyze CPU utilization data.
Monitoring
Cexporter
Tool used to collect CPU utilization metrics for analysis.
Key Actionable Insights
1Implement a feedback loop for CPU allocation based on historical usage data to optimize resource distribution across containers.This approach allows for proactive scaling, ensuring that resources are allocated efficiently based on actual workload demands rather than reactive adjustments.
2Utilize a standardized model for CPU allocation to simplify management across different storage technologies.By relying on CPU utilization metrics, teams can avoid the complexities of creating separate models for each technology, thus saving time and reducing errors.
3Monitor peak CPU utilization to ensure adequate resource availability during regional failovers.Setting a target CPU utilization ratio helps maintain performance during unexpected spikes in demand, ensuring that services remain reliable even under stress.
Common Pitfalls
1
Relying solely on manual adjustments for CPU allocations can lead to inefficiencies and reactive scaling.
This often results in overprovisioning or underutilization of resources, which can increase costs and reduce performance reliability.
2
Failing to consider workload variability can lead to inappropriate core allocations.
Workloads can fluctuate significantly, and without a model that accounts for these changes, containers may be consistently over or under-provisioned.
Related Concepts
Vertical Scaling Vs Horizontal Scaling
Container Orchestration Best Practices
Resource Allocation Strategies In Cloud Environments