Overview
The article discusses the evolution of LinkedIn's Endorsements infrastructure, highlighting the need for a more effective system to provide valuable skill endorsements. It outlines the current architecture, the challenges faced, and the proposed improvements through machine learning and backend reengineering.
What You'll Learn
1
How to leverage machine learning for improving endorsement systems
2
Why a structured index is crucial for database performance
3
How to gather and utilize user feedback for product development
Prerequisites & Requirements
- Understanding of machine learning concepts
- Familiarity with database indexing techniques
Key Questions Answered
What are the two main pillars of LinkedIn's Endorsements infrastructure?
The two main pillars are the suggested endorsement pipeline, which generates suggestions for endorsements, and the serving endorsements pillar, which allows members to endorse their connections. These components work together to enhance the endorsement experience for users.
How does LinkedIn optimize its endorsement serving process?
LinkedIn uses a micro-service infrastructure that encapsulates features like endorsements within a single service, allowing for independent evolution. They optimize database performance by heavily indexing endorsements to handle up to 10 billion endorsements and thousands of queries per second.
What is the new target metric for endorsements at LinkedIn?
The new target metric, referred to as 'Highly-Rated Endorsement', focuses on endorsements made by connections who are knowledgeable about both the recipient and the skill. This metric aims to provide meaningful validation of a member's skills.
What feedback mechanism does LinkedIn use for endorsements?
LinkedIn has implemented in-product feedback mechanisms where members can respond to notifications about endorsements. This feedback helps retrain algorithms to improve the endorsement experience based on user preferences and values.
Key Statistics & Figures
Total endorsements given
10 billion
This statistic highlights the scale of LinkedIn's endorsement feature and the engagement it generates among users.
Queries per second (QPS) supported
thousands
This performance metric indicates the capacity of LinkedIn's endorsement infrastructure to handle high traffic efficiently.
Technologies & Tools
Data Processing
Hadoop
Used for generating suggestions in the endorsement pipelines through machine learning workflows.
Key-value Store
Voldemort
Utilized for storing and querying endorsement suggestions.
Key Actionable Insights
1Implement a structured indexing strategy for your database to enhance query performance.By organizing data hierarchically, as LinkedIn does with endorsements, you can significantly reduce query times and improve overall application responsiveness.
2Utilize user feedback to refine product features and algorithms.Gathering direct feedback from users can provide valuable insights that help tailor your product to better meet user needs, as demonstrated by LinkedIn's approach to endorsements.
3Consider the value of endorsements based on the relationship between endorser and recipient.Understanding the dynamics of professional relationships can lead to more meaningful endorsements, enhancing the perceived value of your endorsement system.
Common Pitfalls
1
Relying solely on quantity over quality in endorsements can diminish their value.
Focusing on the total number of endorsements rather than the quality and relevance of each endorsement can lead to user disengagement and a lack of trust in the system.
Related Concepts
Machine Learning
Data Management
Product Design