How Pinterest Uses Apache Kafka
14 engineering articles about Apache Kafka from Pinterest's engineering team
Other Pinterest Technologies
Other Companies Using Apache Kafka
Articles
Filter:
The article discusses Pinterest's transition to a next-generation database ingestion framework designed to address the limitations of legacy systems.
Pinterest Engineering
10 min read
Includes Code
Has Summary
--
The article discusses Change Data Capture (CDC) at Pinterest, detailing its importance for real-time data processing and the implementation of a Generic CDC solution using Debezium.
Pinterest Engineering
8 min read
Has Summary
--
The article discusses Pinterest's implementation of Tiered Storage for Apache Kafka®️, highlighting a broker-decoupled approach that offloads data to cheaper remote storage.
Pinterest Engineering
24 min read
Includes Code
Has Summary
--
The article discusses the implementation and operational benefits of the Unified PubSub Client (PSC) at Pinterest, highlighting improvements in developer velocity, stability, and scalability.
Pinterest Engineering
11 min read
Includes Code
Has Summary
--
The article discusses the development of a unified PubSub client library at Pinterest, aimed at improving the scalability, stability, and developer velocity of the Logging Platform.
Pinterest Engineering
12 min read
Includes Code
Has Summary
--
The article discusses Pinterest's automated campaign budget optimization product, which helps advertisers efficiently allocate their advertising budgets across various ad groups.
Pinterest Engineering
10 min read
Has Summary
--
The article discusses the Campaign Budget Optimization (CBO) product developed by Pinterest's Ads Intelligence team, which automates the distribution of advertising budgets across ad groups to maxi...
Pinterest Engineering
8 min read
Has Summary
--
MemQ is a new, efficient, and scalable cloud-native PubSub system developed by Pinterest, designed to handle Near Real-Time data transportation while being up to 90% more cost-effective than Apache...
Pinterest Engineering
12 min read
Has Summary
--
The article discusses Pinterest's transition from Lambda architecture to Kappa architecture for visual signals infrastructure, focusing on the need for real-time processing of machine learning sign...
Pinterest Engineering
8 min read
Has Summary
--
This article discusses how Pinterest's Logging Platform team utilizes graph algorithms to optimize Kafka operations, particularly focusing on addressing the imbalanced leader problem in Kafka clust...
Pinterest Engineering
6 min read
Has Summary
--
Pinterest operates one of the largest Kafka deployments in the cloud, utilizing Apache Kafka as a message bus for data transport and real-time streaming services.
Pinterest Engineering
6 min read
Has Summary
--
The article discusses the development of Pin stats at Pinterest, a tool designed to provide businesses with near-real-time analytics on their Pins.
Pinterest Engineering
6 min read
Has Summary
--
The article discusses how Pinterest utilizes the Kafka Streams API to create a predictive budgeting system that minimizes ad overdelivery.
Pinterest Engineering
7 min read
Has Summary
--
The article discusses Pinterest's implementation of a real-time data pipeline for analytics, leveraging technologies like Spark Streaming and MemSQL.
Pinterest Engineering
4 min read
Has Summary
--
You've reached the end! All 14 articles loaded.