Overview
The article discusses the Mobile Test Orchestrator (MTO), a framework developed by LinkedIn to efficiently manage and execute Android tests across multiple emulators and servers. It highlights the challenges of scaling test execution while maintaining reasonable run times, and presents MTO as a solution to enhance productivity in continuous integration pipelines.
What You'll Learn
1
How to distribute Android tests across multiple emulators to reduce execution time
2
Why using a pull model for test distribution enhances resource utilization
3
How to leverage Python's multiprocessing and asyncio for orchestrating tests
Prerequisites & Requirements
- Understanding of Android testing principles and frameworks
- Familiarity with Android Debug Bridge (ADB)(optional)
- Experience with Python programming
Key Questions Answered
How does the Mobile Test Orchestrator improve Android testing efficiency?
The Mobile Test Orchestrator (MTO) improves efficiency by distributing tests across multiple emulators and servers, significantly reducing test execution times. It allows for concurrent execution of tests, which helps maintain a target run time of 10 minutes or less, even as the number of tests grows into the thousands.
What are the key differences between concurrency and parallelism in the context of MTO?
Concurrency in MTO refers to managing multiple tasks efficiently, while parallelism involves executing multiple tasks simultaneously using more resources. MTO utilizes both concepts by running tests on multiple emulators in parallel while orchestrating the test distribution using a single core for concurrency.
What results were achieved by implementing the Mobile Test Orchestrator?
After implementing MTO, LinkedIn reduced the test execution time from approximately 80 minutes on a single machine to nearly 10 minutes by distributing tests across 10 hosts, each running 16 emulators. This scalability allows for future growth in testing capacity.
What are the advantages of using a pull model for test execution in MTO?
The pull model allows for strategic ordering of test execution, ensuring longer-running tests are prioritized. It also enhances robustness against failures, as remaining worker nodes can continue pulling tests from the queue if one fails, minimizing disruption.
Key Statistics & Figures
Initial test execution time
80 minutes
This was the execution time for tests on a single machine with 16 emulators before implementing MTO.
Target test execution time
10 minutes
MTO aims to keep test execution times at or below this target to enhance developer productivity.
CPU utilization of orchestration software
1-2%
This low CPU usage indicates that most resource consumption occurs in the emulators during test execution.
CPU utilization by emulators
200%+
Emulators can utilize more than 2 fully dedicated cores when running tests, demonstrating their resource-intensive nature.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
Python
Used for developing the orchestration software in MTO, leveraging its multiprocessing and asyncio capabilities.
Tools
Android Debug Bridge (adb)
Facilitates communication and command execution between the host and emulators during testing.
Key Actionable Insights
1Implement the Mobile Test Orchestrator to manage Android test distribution effectively.Using MTO can significantly reduce test execution times, allowing developers to receive faster feedback and maintain productivity as the number of tests increases.
2Utilize Python's multiprocessing and asyncio modules to enhance the orchestration of distributed tests.These tools facilitate efficient management of I/O operations and resource allocation, which are crucial for running tests across multiple emulators.
3Adopt a pull model for test distribution to improve resource utilization and manage test execution order.This approach helps in prioritizing longer tests and provides resilience against worker node failures, ensuring smoother test operations.
Common Pitfalls
1
Overloading a single host with too many emulators can lead to increased startup times and resource contention.
It's essential to balance the number of emulators per host to maintain efficient test execution without overwhelming the system.
Related Concepts
Distributed Testing Frameworks
Continuous Integration And Delivery
Concurrency And Parallelism In Software Development