Overview
The article discusses the 'Quick Deploy' project at LinkedIn, which utilizes a distributed systems approach to enhance developer productivity by allowing engineers to deploy only the services they are modifying locally while keeping the rest in a shared environment. It outlines the design, implementation challenges, and benefits of this system.
What You'll Learn
1
How to set up a routing profile for local service deployment
2
Why using a proxy server is essential for routing requests in a distributed system
3
How to avoid data contention issues by creating unique test accounts
4
When to deploy both asynchronous technologies and relevant service clusters locally
Key Questions Answered
How does Quick Deploy improve developer productivity at LinkedIn?
Quick Deploy allows developers to deploy only the services they are modifying locally, which significantly reduces the time needed to set up their development environment. This approach also enables engineers to work with larger datasets early in the development cycle, catching performance issues sooner.
What challenges did LinkedIn face when implementing Quick Deploy?
LinkedIn encountered issues such as shared database access among engineers leading to unexpected behavior and problems with asynchronous communication where local and external services could conflict. Solutions included creating unique test accounts and deploying necessary asynchronous technologies locally.
What is the developer workflow for using Quick Deploy?
The developer workflow involves setting up a routing profile that specifies which services are deployed locally, using a proxy server to test, and ensuring that all requests are routed correctly based on the routing profile stored in cookies.
Why is it important to use a proxy in Quick Deploy?
The proxy is crucial for routing requests correctly between locally deployed services and those in the shared environment, ensuring that inter-service requests are handled appropriately based on the developer's routing profile.
Technologies & Tools
Messaging
Activemq
Used for asynchronous communication between services.
Scheduling
Quartz
Used for performing scheduled tasks in the system.
Key Actionable Insights
1Implement a routing profile system to streamline local deployments.By allowing developers to specify which services they need locally, teams can reduce setup time and focus on the code that matters, improving overall efficiency.
2Create unique test accounts for each engineer to avoid data conflicts.This practice prevents unexpected changes in shared test data from affecting multiple developers, leading to a smoother testing process.
3Deploy asynchronous technologies locally to manage message consumption effectively.This ensures that local services do not contend with external services for message processing, reducing unpredictability in behavior.
Common Pitfalls
1
Sharing the same database among engineers can lead to unexpected behavior.
When multiple developers use the same test accounts, changes made by one can affect others, causing confusion and errors during testing.
2
Asynchronous communication can lead to contention issues between local and external services.
Without proper deployment strategies, services may conflict over message consumption, leading to unpredictable outcomes.
Related Concepts
Distributed Systems
Developer Experience
Service Deployment Strategies