The first in a series on scaling service development, this article looks at the core structure, the Service IDL, underpinning the new…
Overview
This article discusses Airbnb's transition to a service-oriented architecture (SOA) and introduces the Service IDL (Interface Definition Language) as a key component for scaling service development. It highlights the challenges faced during this transition and outlines the approach taken to enhance service robustness, performance, and maintainability.
What You'll Learn
1
How to implement Thrift as a Service IDL for Java services
2
Why using a strong-typed API improves service development
3
How to transition from JSON-over-HTTP to Thrift-over-HTTP without disrupting existing services
4
When to use automated RPC client generation to save engineering time
Prerequisites & Requirements
- Familiarity with service-oriented architecture concepts
- Experience with Dropwizard framework(optional)
Key Questions Answered
What challenges does Airbnb face in transitioning to a service-oriented architecture?
Airbnb faces challenges such as a lack of clearly defined service interfaces, the need for extensive engineering work to implement RPC clients, and inefficiencies in JSON request/response payloads. These issues can lead to production incidents and increased development time.
How does Thrift improve service development at Airbnb?
Thrift provides a strong-typed API that enforces clear definitions for service interfaces and data schemas, simplifying service development. This allows engineers to focus on business logic rather than plumbing and monitoring tasks, ultimately increasing development velocity.
What is the role of the 'make-me-a-service' tool in Airbnb's service development?
The 'make-me-a-service' tool helps engineers bootstrap a ready-to-deploy Dropwizard application skeleton, allowing them to quickly add RESTful and JSON-over-HTTP service resource endpoints. This tool standardizes server-side best practices and accelerates the development process.
How does Airbnb ensure backward compatibility in its services?
Airbnb ensures backward compatibility by using Thrift's built-in schema enforcement and generating server-side and client-side code with API changes in mind. This structured approach simplifies managing API versions and maintaining service stability.
Key Statistics & Figures
Time saved per Java service
2-3 weeks
This estimate reflects the reduction in engineering time when using the new service IDL-driven Java service development process compared to the previous method.
Technologies & Tools
Backend
Thrift
Used as the Service IDL to define service APIs and facilitate RPC client generation.
Backend
Dropwizard
Framework used for building backend services at Airbnb.
Key Actionable Insights
1Implementing Thrift as a Service IDL can significantly streamline your service development process.By defining service APIs in Thrift, you can automate client generation and enforce strong typing, which reduces errors and speeds up development.
2Transitioning to Thrift-over-HTTP can improve performance while maintaining compatibility with existing services.This approach allows you to leverage the benefits of Thrift's binary transport for performance while still supporting JSON for legacy services.
3Utilizing automated RPC client generation saves valuable engineering time and ensures consistency across different language implementations.This reduces the overhead of maintaining multiple client libraries and helps keep them in sync, minimizing the risk of discrepancies.
Common Pitfalls
1
Failing to define clear service interfaces can lead to confusion and production incidents.
Without a strong-typed API, developers may struggle to understand how to interact with services, leading to errors and increased debugging time.
2
Neglecting to implement standard practices for request context propagation can complicate monitoring and debugging.
Inconsistent handling of request contexts across services can make it difficult for Site Reliability Engineers (SREs) to trace issues and maintain service reliability.
Related Concepts
Service-oriented Architecture
API Design Principles
Inter-service Communication Patterns