Overview
The article discusses how Pinterest employs machine learning techniques to deliver relevant ads to users by utilizing unique first-party signals. It highlights the implementation of Actalike audiences and the combination of user embeddings with multi-layer perceptron classifiers to enhance audience expansion and ad relevance.
What You'll Learn
1
How to combine user embeddings with MLP classifiers for ad targeting
2
Why sample weighting can improve model performance in audience expansion
3
How to evaluate audience expansion models using recall and precision metrics
Prerequisites & Requirements
- Understanding of machine learning concepts and audience targeting
- Familiarity with Spark and Kubernetes(optional)
Key Questions Answered
How does Pinterest use machine learning to deliver relevant ads?
Pinterest utilizes machine learning models that combine user embeddings with multi-layer perceptron classifiers to effectively target ads based on user interests and behaviors. This approach enhances audience expansion by leveraging first-party signals to ensure ads are relevant and non-intrusive.
What are the advantages of using a combined NN approach for audience expansion?
The combined NN approach outperforms traditional regression-based and similarity-based methods by leveraging the strengths of both. It allows for faster convergence and better performance across different seed list sizes, improving recall and precision metrics significantly.
What metrics are used to evaluate audience expansion models?
The evaluation of audience expansion models is conducted using recall@k and precision@k metrics. These metrics assess how well the model ranks similar users against a seed list, providing insights into the effectiveness of the audience expansion.
What improvements were observed during the online A/B tests?
In online A/B tests, the combined NN model led to a 3.6% increase in ad impressions and a 3.1% increase in revenue, despite a 5.7% drop in click-through rate. This indicates improved user engagement quality with the ads served.
Key Statistics & Figures
Increase in ad impressions
3.6%
Observed during the online A/B tests comparing the combined NN model with the blended model.
Increase in revenue
3.1%
Also noted during the online A/B tests for the combined NN model.
Drop in click-through rate (CTR)
5.7%
This occurred during the online A/B tests, indicating a shift in user engagement quality.
Drop in hide rate (HDR)
8.3%
This metric improved, suggesting better user engagement with ads.
Gain in good click ratio (GCR)
2.7%
Indicates improved quality of user engagement with ads served.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
Spark
Used for scaling the system to support regularly trained models.
Backend
Kubernetes
Utilized for managing containerized applications and scaling the ad targeting system.
Key Actionable Insights
1Implementing a combined NN approach can significantly enhance ad targeting effectiveness.By integrating user embeddings with MLP classifiers, advertisers can achieve better audience expansion results, which is crucial for maximizing ad relevance and user engagement.
2Utilizing sample weighting based on user engagement metrics can improve model performance.Incorporating engagement metrics such as click-through rates into model training allows for a more nuanced understanding of user interactions, leading to better targeting outcomes.
3Regularly evaluate audience expansion models using recall and precision metrics.These metrics provide a clear indication of model performance, helping to ensure that the audience expansion strategies remain effective and aligned with business goals.
Common Pitfalls
1
Relying solely on regression-based models can limit audience expansion effectiveness.
Regression-based models may not capture the full complexity of user interactions, leading to suboptimal targeting results. Combining different approaches can mitigate this issue.
Related Concepts
Machine Learning In Advertising
Audience Targeting Strategies
User Engagement Metrics
Neural Network Applications In Marketing