How Pinterest Uses Kubernetes

The article reflects on a decade of AI platform development at Pinterest, detailing the evolution from fragmented machine learning stacks to a unified AI platform that supports various models.

The article discusses Pinterest's transition to Moka, a next-generation data processing platform built on AWS Elastic Kubernetes Service (EKS).

The article discusses Pinterest's journey in enhancing developer experience through the creation of PinConsole, an Internal Developer Platform built on Backstage.

This article details the challenges faced by Pinterest during the migration of its search infrastructure, Manas, to Kubernetes.

This article discusses Pinterest's transition from a Hadoop-based platform to a Kubernetes-based data processing solution named Moka.

The article discusses the transition from Apache Hadoop YARN to Apache YuniKorn for resource management in Pinterest's batch processing platform, Monarch, now rebranded as Moka.

The article discusses Pinterest's adoption of TiDB as a replacement for HBase, detailing the motivations, selection methodology, and the journey of integrating TiDB into their infrastructure.

The article discusses the integration of Ray infrastructure at Pinterest, detailing the journey, challenges, and solutions implemented to optimize machine learning workflows.

In 2023, Pinterest Engineering achieved significant milestones by launching various innovative products, including new ad formats and inclusive AI technology.

PinCompute is a Kubernetes-backed general-purpose compute platform developed by Pinterest to enhance application management and efficiency.

The article discusses MLEnv, Pinterest's standardized ML engine that has significantly improved machine learning (ML) development and innovation within the company.

This article discusses Pinterest's implementation of a finer-grained access control (FGAC) framework to manage data access securely and efficiently within their data engineering platform.

The article discusses the journey of the Cloud Runtime team at Pinterest in enhancing the performance of their Kubernetes control plane, achieving a Service Level Objective (SLO) increase from 99% ...

The article discusses Pinterest's migration from its legacy workflow system, Pinball, to a new platform called Spinner, built on Apache Airflow.

The article discusses Spinner, Pinterest's workflow platform, detailing its evolution from an in-house scheduler called Pinball to Apache Airflow.

The article summarizes the most popular engineering blog posts from Pinterest in 2021, highlighting significant advancements and initiatives by Pinterest engineers.

The article discusses how Pinterest employs machine learning techniques to deliver relevant ads to users by utilizing unique first-party signals.

The article discusses Pinterest's journey in scaling its Kubernetes platform, focusing on governance, resilience, and operability.

This article discusses how Pinterest's Engineering Productivity team redesigned their Continuous Integration (CI) system to achieve over 50% faster build times.

The article discusses how Pinterest empowered its data scientists and machine learning engineers by building a PySpark infrastructure that addresses challenges faced with existing tools like Hive a...

The article discusses PinText, a multitask text embedding system developed by Pinterest to improve the representation of textual data in their platform.

This article discusses Pinterest's journey in building a Kubernetes platform to manage its diverse workloads efficiently.

The article discusses Pinterest's implementation of Presto, an open-source distributed SQL query engine, detailing the challenges faced and solutions developed to manage large-scale data analysis.

The article discusses the open-sourcing of Singer, Pinterest's logging agent designed for efficient and reliable data collection from numerous hosts.

Pinterest operates one of the largest Kafka deployments in the cloud, utilizing Apache Kafka as a message bus for data transport and real-time streaming services.

The article discusses the enhancements made to Knox, Pinterest's open-source secret management service, enabling it to support SPIFFE x509 identity documents for authentication in multi-tenant envi...

Pinterest has officially joined the Cloud Native Computing Foundation (CNCF) to collaborate on cloud-native technologies and share insights from its experience operating at scale in a public cloud ...

The article discusses Pinterest's transition from EC2 instances to Docker containers, detailing the migration process, benefits, and challenges faced by the engineering teams.