PID Controller for Cinnamon

Jakob Holdgaard Thomsen, Vladimir Gavrilenko, Jesper Lindstrom Nielsen, Timothy Smyth

Uber

•

Jakob Holdgaard Thomsen, Vladimir Gavrilenko, Jesper Lindstrom Nielsen, Timothy Smyth

•15 min read•intermediate•

--

•View Original

Overview

This article discusses the implementation of a PID controller in the Cinnamon system at Uber, focusing on how it effectively manages request rejection rates during overload situations. It covers the architecture, functionality, and performance of the PID controller, highlighting its ability to maintain a stable rejection ratio while optimizing resource utilization.

What You'll Learn

1

How to implement a PID controller for managing request rejection rates

2

Why PID controllers are effective in controlling load shedding

3

When to apply PID tuning for optimal performance in microservices

Prerequisites & Requirements

Understanding of PID controllers and their components
Familiarity with microservices architecture(optional)

Key Questions Answered

How does the PID controller in Cinnamon manage request rejection rates?

The PID controller in Cinnamon continuously adjusts the rejection rate based on incoming and outgoing requests. It calculates the P, I, and D terms using the request throughput to maintain a stable rejection ratio, ensuring efficient resource utilization while minimizing queue times.

What are the key components of a PID controller?

A PID controller consists of three components: Proportional (P), which measures the difference between the target and current values; Integral (I), which accumulates past errors; and Derivative (D), which predicts future errors based on the rate of change. In Cinnamon, the D component is not utilized.

What challenges does Cinnamon face in implementing the PID controller?

Cinnamon's main challenge is finding a target function for the PID controller that is portable across various services without requiring configuration changes. This is crucial due to the varying request processing rates across different microservices.

Key Statistics & Figures

Rejection span

around 10 percentage points

This span is consistent across different traffic levels, indicating the PID controller's effectiveness in managing load shedding.

Requests per second (RPS) handled

1,300 RPS

This is the total load capacity of the test service used in the experiments.

Key Actionable Insights

1
Implementing a PID controller can significantly enhance the stability of request handling in microservices.
By dynamically adjusting the rejection rate based on real-time traffic patterns, services can maintain optimal performance even during overload situations.

2
Tuning the Kp and Ki constants is essential for achieving desired performance outcomes.
The article highlights that Kp = 0.1 and Ki = 1.4 were found to be effective, indicating the importance of empirical testing in PID tuning.

Common Pitfalls

1

Failing to properly tune the PID constants can lead to oscillations in request handling.

Without careful tuning, the system may either reject too many requests or allow excessive queuing, negatively impacting performance.

Related Concepts

Pid Controllers

Load Shedding

Microservices Architecture