Test Generation and CI
Overview
This article discusses the final installment of Airbnb's series on Android testing, focusing on test generation and continuous integration (CI) setup. It details the tooling used to automate test creation and the CI infrastructure that supports efficient testing workflows.
What You'll Learn
1
How to generate JUnit test files automatically for MvRx Fragments using Kotlin scripting
2
Why detecting changed fragments in CI can optimize test execution time
3
How to implement test sharding using Flank to improve testing efficiency
Prerequisites & Requirements
- Understanding of Kotlin and Android testing frameworks
- Familiarity with CI/CD tools like Buildkite and Firebase(optional)
Key Questions Answered
How does Airbnb automate the generation of test files for Android Fragments?
Airbnb uses a Kotlin script to parse the app directory, extracting names of MvRx Fragments and generating JUnit test files automatically. This approach reduces manual effort and ensures that tests are consistently created for each Fragment, enhancing the testing process.
What strategies does Airbnb use to optimize CI for Android testing?
Airbnb's CI pipeline runs on every GitHub Pull Request, utilizing a test generation script to include only affected Fragments. This selective testing minimizes unnecessary test runs, saving time and resources, particularly on Firebase costs.
What is the role of Flank in Airbnb's testing strategy?
Flank is used to shard tests across multiple Firebase test matrices, allowing tests to run in parallel. This significantly reduces the overall test execution time from about two hours to just a few minutes, improving efficiency in the CI process.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Programming Language
Kotlin
Used for scripting the test generation process.
Testing Framework
Flank
Used for sharding tests to run in parallel on Firebase.
Cloud Service
Firebase
Used for running integration tests.
Testing Framework
Junit
Framework for writing and running tests.
Visual Testing Tool
Happo
Used for generating visual diff reports.
Key Actionable Insights
1Implementing automated test file generation can significantly streamline your testing process.By using scripts to generate tests based on your code structure, you can reduce manual overhead and ensure comprehensive coverage of your components.
2Optimizing your CI pipeline to only run tests on changed code can lead to faster feedback loops.This approach not only saves time but also reduces costs associated with testing infrastructure, making it a valuable strategy for teams looking to enhance productivity.
3Utilizing test sharding with tools like Flank can drastically improve test execution times.By running tests in parallel, teams can handle larger test suites without the long wait times typically associated with sequential test execution.
Common Pitfalls
1
Failing to account for module dependencies can lead to incomplete test coverage.
If a module that depends on another changed module is not included in the tests, it can result in undetected issues. It's crucial to maintain an accurate module dependency graph to ensure all affected modules are tested.
2
Not handling Firebase outages can confuse developers during test failures.
Automating checks for Firebase status before running tests can prevent unnecessary confusion and support better developer experience by providing clear instructions during outages.
Related Concepts
Automated Testing Frameworks
Continuous Integration Practices
Test-driven Development
Mocking Frameworks