Overview
This article discusses Pinterest's journey in building a Kubernetes platform to manage its diverse workloads efficiently. It highlights the challenges faced by the infrastructure team and the solutions implemented, including custom resource definitions (CRDs) tailored to Pinterest's specific needs.
What You'll Learn
1
How to implement Pinterest-specific Custom Resource Definitions (CRDs) in Kubernetes
2
Why container orchestration is essential for managing diverse workloads
3
When to use CRDs versus other Kubernetes resource management techniques
Prerequisites & Requirements
- Understanding of Kubernetes and container orchestration concepts
- Familiarity with CLI tools for Kubernetes management(optional)
Key Questions Answered
What challenges did Pinterest face in managing workloads before Kubernetes?
Pinterest faced multiple challenges, including a lack of unified experience for engineers when launching workloads, significant maintenance burdens from engineers managing their own VM fleets, and difficulties in building infrastructure governance tools due to separated management systems.
How does Pinterest customize Kubernetes for its specific needs?
Pinterest has developed Custom Resource Definitions (CRDs) that bundle various native Kubernetes resources together, inject necessary runtime support, and manage lifecycle for native resources. This approach simplifies the deployment process for engineers and reduces errors.
What are the main types of Custom Resource Definitions (CRDs) used at Pinterest?
Pinterest has designed several CRDs including PinterestService for stateless services, PinterestJobSet for batch jobs, PinterestCronJob for periodic workloads, PinterestDaemon for infrastructure daemons, and PinterestTrainingJob for machine learning tasks, each tailored to specific workflow requirements.
What is the application deployment workflow at Pinterest using Kubernetes?
Developers interact with the Kubernetes cluster via CLI and UI, submitting workload configurations through a Job Submission Service that validates and transforms Pinterest CRDs into native Kubernetes resources, ensuring necessary runtime support is included before deployment.
Key Statistics & Figures
Number of Pinners
300 million
Pinterest serves a vast user base of 300 million Pinners who have saved over 200 billion Pins.
Number of Pins
200 billion
The platform hosts more than 200 billion Pins across over 4 billion boards.
Number of services developed
thousands
Pinterest has developed thousands of services ranging from microservices to large monolithic services.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Container Orchestration
Kubernetes
Used to manage diverse workloads and unify workload management at Pinterest.
Containerization
Docker
Used to dockerize most of Pinterest's production workloads.
Key Actionable Insights
1Implementing Custom Resource Definitions (CRDs) can significantly streamline workload management in Kubernetes.By bundling multiple Kubernetes resources into a single CRD, teams can reduce complexity and improve deployment efficiency, making it easier for developers to focus on application logic rather than infrastructure details.
2Centralizing infrastructure management through Kubernetes can alleviate maintenance burdens on engineering teams.This approach allows for easier upgrades and management of resources, which can lead to improved service reliability and developer productivity.
3Utilizing a Job Submission Service can enhance the deployment workflow by enforcing validation and ensuring only reviewed workloads are submitted.This practice not only improves security but also helps maintain consistency across deployments, reducing the likelihood of errors.
Common Pitfalls
1
Failing to properly configure ingress can lead to service endpoint issues.
This often occurs when developers overlook ingress settings, resulting in misconfigured endpoints that disrupt service availability.
2
Over-reliance on templating systems like Helm can lead to errors in diverse workloads.
Using templating systems may not accommodate the unique needs of various workloads, leading to complications in continuous deployment processes.
Related Concepts
Container Orchestration
Custom Resource Definitions (crds)
Kubernetes Deployment Strategies
Infrastructure Governance