Learn about the performance findings from months of repo build testing and tips we learned along the way.
Overview
The article discusses the performance benefits and debugging techniques related to Copy-on-Write (CoW) linking in the context of Dev Drive, which was released in Windows 11 and will be part of Windows Server 2025. It provides insights from performance testing, methods to identify CoW links, and practical tips for developers using Dev Drive.
What You'll Learn
1
How to measure the performance impact of Copy-on-Write linking in your builds
2
Why certain codebases benefit more from CoW linking than others
3
How to identify CoW links using the fsutil command
4
When to use ProcMon and Xperf for performance monitoring with Dev Drive
Prerequisites & Requirements
- Understanding of MSBuild and project dependencies
- Familiarity with command-line tools like fsutil(optional)
Key Questions Answered
What performance improvements can be expected from using Copy-on-Write linking?
Performance testing showed that many repositories experienced a reduction of 10% or more in build times, with some achieving up to a 43% improvement. The benefits were particularly pronounced in C# projects with deep dependencies and in repositories that required extensive copying for microservice layouts.
How can developers determine if a file is a CoW link?
Developers can use the fsutil command with the 'queryExtentsAndRefCounts' subcommand to check if a file is a CoW link. This command provides details about the reference counts for the file, indicating how many block clones exist on the disk.
What tools can be used for monitoring performance with Dev Drive?
ProcMon and Microsoft Performance Recorder (Xperf) are recommended tools for monitoring performance. Developers can attach filter drivers to these tools to analyze file operations and performance metrics effectively.
What should be done if too many CoW references are leaked?
If a file fails to copy due to too many clones, developers can use the refsutil command to scan and fix dangling references. This tool helps manage the maximum limit of 8176 clones per data block in Dev Drive.
Key Statistics & Figures
Maximum performance improvement
43%
Achieved in some repositories during performance testing with CoW linking.
Minimum performance improvement
10%
Observed in many repositories after implementing CoW linking.
Clone limit for a data block
8176 clones
Maximum number of clones allowed in Dev Drive based on ReFS.
Technologies & Tools
Storage
Dev Drive
Utilized for enhanced performance in build processes through CoW linking.
SDK
Microsoft.build.copyonwrite
Provides CoW functionality for MSBuild projects.
Monitoring
Microsoft Performance Recorder (xperf)
Used for performance measurement and analysis with Dev Drive.
Key Actionable Insights
1Integrate the CopyOnWrite SDK into your MSBuild repository to leverage CoW benefits.By integrating this SDK, developers can enhance build performance, particularly for C# projects with complex dependencies, leading to faster build times.
2Regularly monitor your Dev Drive using ProcMon to identify performance bottlenecks.Using ProcMon allows developers to track file operations and understand how CoW linking affects their build processes, enabling proactive performance tuning.
3Utilize the fsutil command to identify CoW links and manage file references effectively.This command helps developers understand the underlying file structure and reference counts, which is crucial for optimizing storage and performance.
4Be proactive in managing CoW references to avoid hitting the clone limit.By regularly cleaning up orphaned references, developers can prevent issues related to excessive clones, ensuring smoother build processes.
Common Pitfalls
1
Failing to monitor CoW references can lead to hitting the clone limit, causing build failures.
This happens when developers do not regularly clean up orphaned references, leading to excessive clones that exceed the allowed limit.
2
Not utilizing the performance monitoring tools effectively can result in missed optimization opportunities.
If developers do not use tools like ProcMon or Xperf, they may overlook critical performance bottlenecks that could be addressed for better build efficiency.
Related Concepts
Copy-on-write Linking
Msbuild Performance Optimization
File System Monitoring Techniques