Airbnb Categories Blog Series — Part II : ML Categorization
Overview
This article discusses the process of building categories for Airbnb listings using a combination of machine learning (ML) and human review. It details the methodologies for defining categories, generating listing candidates, and the integration of ML models to enhance the categorization process.
What You'll Learn
1
How to define product-driven categories for listings
2
How to leverage multiple data signals for ML model training
3
How to implement a human review process to enhance ML categorization
4
Why combining ML models with human review improves accuracy
Key Questions Answered
How does Airbnb define categories for its listings?
Airbnb defines categories through product-driven definitions, such as specifying that the 'Lakefront' category includes listings within 100 meters of a lake. This involves leveraging various structured and unstructured data signals to accurately categorize listings.
What types of data signals are used for listing categorization?
Data signals for categorization include host-provided listing information, guest reviews, host guidebooks, and external data such as satellite and climate data. These signals help in generating candidates for categories with higher precision.
What role does human review play in the categorization process?
Human review is crucial as it validates the categorization of listings, selects cover photos, and assesses listing quality. This process allows for the refinement of ML models based on real feedback, enhancing overall accuracy.
How are ML models trained for categorization tasks?
ML models are trained using XGBoost binary classification techniques, leveraging agent category assignments as labels and various listing signals as features. This allows for precise control over candidate selection based on model score thresholds.
Key Statistics & Figures
Average Precision improvement
23%
The ML model improved Average Precision from 0.74 to 0.91 by incorporating additional signals beyond just POI data.
Top 3 precision of cover photo selection
70%
The Vision Transformer model achieved an average Top 3 precision of 70% for selecting appropriate cover photos.
Human review time improvement
18%
The use of the Vision Transformer model sped up the human review process by 18%.
Technologies & Tools
ML Framework
Xgboost
Used for training binary classification models for listing categorization.
ML Model
Vision Transformer
Utilized for selecting the most appropriate cover photos based on category context.
Key Actionable Insights
1Implement a structured approach to define categories based on user needs and data signals.By clearly defining categories, you can enhance user experience and improve searchability of listings, leading to higher engagement and satisfaction.
2Utilize both ML and human review processes to refine categorization accuracy.Combining automated ML processes with human oversight ensures that the categorization remains relevant and accurate, which is vital for maintaining user trust.
3Incorporate diverse data sources to enrich the feature set for ML models.Using a variety of data signals, including external sources, can significantly improve model performance and lead to better categorization outcomes.
Common Pitfalls
1
Over-reliance on dominant features can lead to overfitting in ML models.
This can occur if the model is trained primarily on a single feature, such as distance from a POI, without considering other relevant signals. To avoid this, implement techniques like feature dropout during training.
Related Concepts
Machine Learning In Categorization
Human-in-the-loop Systems
Data Signal Integration For ML