Overview
The article discusses the development of Aperture, a real-time user action counting system for ads at Pinterest. It highlights the challenges of counting user interactions, the architecture of the system, and the rationale behind using a centralized service for tracking user actions.
What You'll Learn
1
How to implement a real-time user action counting system for ads
2
Why frequency control is crucial for ad delivery
3
How to design a flexible query interface for counting events
Prerequisites & Requirements
- Understanding of real-time data processing and event tracking
- Familiarity with data storage solutions like RocksDB(optional)
Key Questions Answered
What is the purpose of the Aperture system in Pinterest's ad infrastructure?
Aperture serves as a centralized data storage and online events tracking service designed to meet the user action counting requirements for ads. It enables real-time access to user action counts, supporting features like frequency control and fatigue modeling to optimize ad delivery.
How does the frequency control mechanism work in the ad system?
Frequency control manages how often an ad is shown to a Pinner by tracking impression counts over specified time frames. This helps prevent ad fatigue and ensures that users are not overexposed to the same ads, thereby enhancing brand awareness.
What are the two approaches to counting user actions discussed in the article?
The article discusses two approaches: client-side events counting, where ads servers fetch raw events and compute counts ad-hoc, and an independent counting service that provides a dedicated counting layer with generic APIs for serving counts. The latter is preferred for its efficiency.
What challenges does the counting service face regarding event deduplication?
The counting service must ensure that the same user action on a promoted Pin is counted only once within a specified time range. This requires tracking raw events to deduplicate them, as the logging system does not guarantee exactly-once message delivery.
Key Statistics & Figures
P99 latency for counting service
less than 8ms
This performance metric is crucial for ensuring that the counting service can handle peak loads of 200k queries per second.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
Aperture
An in-house service designed for time series data storage and online event tracking.
Database
Rocksdb
Used as the storage engine for Aperture to handle low-latency event appending and data retrieval.
Message Broker
Kafka
Utilized for logging user action events and managing data flow to the counting service.
Key Actionable Insights
1Implement a centralized counting service to improve efficiency in tracking user actions.Using a dedicated counting service like Aperture can streamline the process of counting user interactions, reducing the complexity of client-side logic and improving response times.
2Utilize frequency control to enhance ad performance and user experience.By managing how often ads are shown to users, advertisers can avoid ad fatigue and maintain engagement, which is critical for maximizing click-through rates.
3Design a flexible query interface to accommodate various counting scenarios.A flexible query interface allows for efficient data retrieval across different dimensions and time ranges, which is essential for adapting to changing business needs and user behaviors.
Common Pitfalls
1
Failing to account for event deduplication can lead to inaccurate user action counts.
Without proper deduplication, repeated actions by users within a short time frame may be counted multiple times, skewing the data and affecting ad performance metrics.
2
Overloading the system with unnecessary data requests can degrade performance.
It's important to optimize the number of requests made to the counting service by only querying for relevant data, which helps maintain low latency and high throughput.
Related Concepts
Real-time Data Processing
Event Tracking Systems
Ad Delivery Optimization
User Engagement Metrics