Overview
The article introduces shiv, an open-source tool developed by LinkedIn for creating single binary artifacts from Python projects that include all dependencies. It addresses performance issues associated with existing tools like PEX by avoiding the use of pkg_resources, thereby optimizing command-line utility execution.
What You'll Learn
1
How to create single binary artifacts from Python projects using shiv
2
Why avoiding pkg_resources can improve CLI invocation speed
3
When to use shiv over other packaging tools like PEX
Prerequisites & Requirements
- Basic understanding of Python packaging and command-line utilities
Key Questions Answered
What is shiv and how does it improve Python packaging?
Shiv is a tool that creates single binary artifacts from Python projects, packaging all dependencies without relying on pkg_resources. This approach significantly enhances CLI invocation speed compared to tools like PEX, which can slow down execution due to dependency management overhead.
How does shiv work to optimize Python application execution?
Shiv works by creating a zipapp that contains the Python project and its dependencies, along with a bootstrap module that injects these dependencies at runtime. This method minimizes latency and avoids the performance penalties associated with pkg_resources.
Why was shiv developed instead of continuing with PEX?
Shiv was developed to address performance issues caused by pkg_resources in PEX. As LinkedIn's tools evolved, the need for a faster invocation time led to the decision to create shiv, which eliminates the reliance on pkg_resources.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Programming Language
Python
Used for developing command-line utilities and libraries at LinkedIn.
Packaging Tool
Pex
An earlier tool used for packaging Python dependencies before shiv was developed.
Key Actionable Insights
1Consider using shiv for packaging Python command-line utilities to improve execution speed.By avoiding pkg_resources, shiv allows for faster command-line interface performance, making it ideal for environments where speed is critical.
2Leverage shiv's ability to package an entire site-packages directory for seamless dependency management.This method ensures that all dependencies are included and accessible, reducing the likelihood of runtime errors due to missing packages.
Common Pitfalls
1
Relying on pkg_resources can lead to significant performance issues in command-line applications.
This happens because pkg_resources scans the entire sys.path at import time, which can slow down the startup time of applications. Avoiding this library can mitigate such delays.
Related Concepts
Python Packaging
Command-line Utilities
Dependency Management