The fourth post of our series on scaling service development, focused on Schema Based Testing Infrastructure.
Overview
This article discusses the implementation of Schema Based Testing Infrastructure at Airbnb, focusing on the challenges of microservices testing and how a schema-oriented approach can enhance service development. It introduces tools and frameworks designed for service owners and consumers to improve API testing and validation processes.
What You'll Learn
1
How to implement Static API Schema Validation to catch breaking changes early
2
Why using a Schema-based API Data Factory simplifies testing processes
3
When to utilize the API Mocking Framework for unit and integration tests
4
How to leverage the API Integration Testing Framework (AIT) for real-time validation
Prerequisites & Requirements
- Understanding of microservices architecture and API concepts
- Familiarity with CI/CD tools and testing frameworks(optional)
Key Questions Answered
What challenges does microservices testing face?
Microservices testing faces challenges such as breaking API changes, lack of usable mock data, and insufficient validation for service owners. These issues can lead to production incidents and hinder engineering productivity.
How does Static API Schema Validation work?
Static API Schema Validation automatically detects bad API schema changes by performing backwards compatibility checks and enforcing schema best practices. It uses an Abstract Syntax Tree (AST) to analyze changes and prevent issues before code merges.
What benefits does the API Mocking Framework provide?
The API Mocking Framework allows for simulated near real-world API interactions in tests, enabling unit and shallow integration tests without setting up dependent services. This reduces complexity and improves testing efficiency.
How does the API Integration Testing Framework (AIT) validate services?
The AIT framework validates services by using predefined test data and ensures semantic correctness of mock data. It allows services to validate themselves without writing boilerplate code, enhancing testing accuracy.
Technologies & Tools
Backend
Thrift
Used as the service IDL to enforce infrastructure standards and engineering best practices.
Key Actionable Insights
1Implementing Static API Schema Validation can significantly reduce the risk of breaking changes in production. By catching issues early in the development cycle, teams can avoid costly rollbacks and maintain service reliability.This approach is particularly useful in environments with frequent API changes, ensuring that all modifications adhere to established schema standards.
2Utilizing the API Data Factory can streamline the process of creating mock data for testing. By defining schema-validated request and response fixtures, teams can reduce redundancy and improve test accuracy.This is beneficial in large teams where multiple services interact, as it ensures consistency in the data used across different testing scenarios.
3Adopting the API Mocking Framework allows for efficient testing without the overhead of setting up dependent services. This can lead to faster feedback loops during development.This is especially relevant for frontend teams that need to test UI components without relying on backend availability.
Common Pitfalls
1
One common pitfall is failing to validate API schema changes before deployment, which can lead to breaking changes in production. This often occurs when teams rely solely on manual code reviews.
To avoid this, implementing automated schema validation tools can help catch issues early and ensure compliance with established standards.
2
Another issue is the duplication of effort in creating mock data for tests, which can lead to inconsistencies and increased maintenance overhead.
Using a centralized API Data Factory can mitigate this problem by providing shared, schema-validated mock data across teams.
Related Concepts
Microservices Architecture
API Design And Testing
Continuous Integration And Continuous Delivery (ci/Cd)
Testing Frameworks And Methodologies