Overview
The article discusses Manas, a high-performing customized search system developed by Pinterest to handle billions of queries and improve search relevance among over 100 billion Pins. It highlights the challenges faced by the previous Lucene-based system and outlines the architecture and features of Manas, which is designed for high performance, availability, and scalability.
What You'll Learn
1
How to implement a customized search system using C++
2
Why reducing serving latency is critical for high query volumes
3
How to optimize indexing for large datasets with inverted and forward indexes
4
When to apply application-specific scoring logic in search systems
Prerequisites & Requirements
- Understanding of search indexing concepts
- Familiarity with C++ programming
Key Questions Answered
What challenges did Pinterest face with its previous search system?
Pinterest's previous Lucene-based search system struggled with increasing query volumes and index sizes, leading to higher latency and limited flexibility for customization. The need for personalized search results and support for complex ranking models further necessitated a new solution, resulting in the development of Manas.
How does Manas improve search performance compared to Lucene?
Manas, built in C++, reduces search backend latency by half and increases capacity by 30% compared to the previous Lucene-based system. It achieves this through a customized architecture that optimizes indexing and serving processes tailored to Pinterest's specific needs.
What is the architecture of the Manas search system?
Manas consists of several stages including query understanding, candidate retrieving, lightweight scoring, full scoring, and blending. It utilizes both inverted and forward indexes to optimize performance and supports application-specific scoring logic for enhanced search relevance.
What is the role of the Leaf service in Manas?
The Leaf service in Manas is responsible for loading index segments, retrieving candidates, and performing both lightweight and full scoring. It is designed to be extensible, allowing for application-specific information to be embedded within the index for customized search results.
Key Statistics & Figures
Monthly mobile text search growth
40 percent
This statistic highlights the increasing reliance on mobile for search on Pinterest.
Visual search growth
60 percent
Indicates a significant rise in the use of visual search features on Pinterest.
Reduction in search backend latency
50 percent
This improvement is a key benefit of transitioning to the Manas system.
Increase in capacity
30 percent
Manas has been designed to handle a greater volume of queries compared to the previous system.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
C++
Used to develop the Manas search system for improved performance.
Backend
Lucene
The previous search system that Manas was built to replace.
Key Actionable Insights
1Implement a customized search system to enhance user experience.By tailoring search algorithms and indexing strategies to specific application needs, developers can significantly improve search relevance and performance, as demonstrated by Manas at Pinterest.
2Optimize indexing processes to handle large datasets efficiently.Utilizing both inverted and forward indexes allows for better data locality and faster query responses, which is crucial for applications with high query volumes.
3Leverage application-specific scoring to improve search results.Embedding application-specific logic within the search framework enables more relevant results tailored to user preferences, enhancing overall satisfaction.
Common Pitfalls
1
Neglecting the importance of indexing optimization can lead to performance bottlenecks.
Without proper indexing strategies, search systems may struggle to handle high query volumes efficiently, resulting in increased latency and poor user experience.
Related Concepts
Search Indexing
Query Optimization
Scalability In Search Systems
Machine Learning In Search Ranking