Learn about how Microsoft manages its internal Engineering Systems infrastructure and is releasing it to third-parties through Managed DevOps Pools.
Overview
The article discusses the challenges faced by Microsoft in managing diverse engineering teams and their infrastructure, leading to the development of 1ES Hosted Pools. This internal service aims to unify CI/CD processes and improve efficiency, which has now evolved into an external offering called Managed DevOps Pools.
What You'll Learn
1
How to reduce CI/CD infrastructure costs by optimizing resource utilization
2
Why using a standardized CI/CD infrastructure improves compliance and security
3
When to implement Managed DevOps Pools for better team mobility and efficiency
Key Questions Answered
What challenges did Microsoft face with its Azure DevOps agents infrastructure?
Microsoft faced challenges such as duplicate efforts among teams, inconsistent support and reliability of CI/CD pipelines, inefficiencies in resource utilization, security risks from non-compliant pools, and difficulties in auditing compliance due to varied practices. These issues stemmed from the diverse needs of over 100,000 engineers working across multiple projects.
How does 1ES Hosted Pools improve CI/CD processes?
1ES Hosted Pools addresses the complexities of managing CI/CD infrastructure by providing a unified service that supports private networking, custom images, business continuity, stateful agents, and flexible SKU options. This standardization allows teams to focus more on development rather than infrastructure maintenance.
What benefits did Microsoft achieve by adopting 1ES Hosted Pools?
Microsoft achieved a 60% reduction in CI/CD infrastructure costs, improved compliance through standardized practices, enhanced developer efficiency by reducing time spent on infrastructure maintenance, and increased team mobility as the number of self-hosted pools decreased significantly.
What is Managed DevOps Pools and how does it relate to 1ES Hosted Pools?
Managed DevOps Pools is an external offering developed from the internal success of 1ES Hosted Pools. It aims to provide similar benefits to external customers, helping them optimize their CI/CD processes while addressing common challenges faced in managing self-hosted agents.
Key Statistics & Figures
Reduction in CI/CD infrastructure costs
over 60%
Achieved by optimizing machine utilization and moving to the most suitable Azure SKU.
Number of self-hosted pools at Microsoft
dropped from 5,000+ to a few dozen pools
This significant reduction indicates improved team mobility and standardization in CI/CD practices.
Technologies & Tools
Devops
Azure Devops
Used for managing CI/CD processes and infrastructure within Microsoft.
Key Actionable Insights
1To enhance CI/CD efficiency, consider adopting Managed DevOps Pools to unify your infrastructure management.This approach minimizes redundant efforts and allows teams to focus on feature delivery rather than infrastructure maintenance, as seen in Microsoft's transition to 1ES Hosted Pools.
2Implement automated scaling features in your CI/CD pipelines to reduce costs and improve resource utilization.Many teams at Microsoft reduced their DevOps bills significantly by optimizing machine usage and leveraging historical data for better decision-making.
3Ensure compliance and security by standardizing your CI/CD infrastructure practices.Using a consistent infrastructure like 1ES Hosted Pools allows for easier auditing and implementation of compliance measures, which is crucial for meeting regulatory requirements.
Common Pitfalls
1
Teams often create redundant tooling for CI/CD processes, leading to wasted development cycles.
This happens when teams do not leverage existing solutions like 1ES Hosted Pools, resulting in inefficiencies and increased costs.
2
Inconsistent support and maintenance of CI/CD infrastructure can lead to prolonged downtimes.
Without a standardized approach, some teams may lack adequate support systems, causing delays in restoring CI/CD pipelines when issues arise.