Overview
The article discusses the challenges and solutions associated with managing external libraries in software development, particularly at LinkedIn. It emphasizes the importance of a reliable Continuous Integration/Continuous Delivery (CI/CD) pipeline and introduces the External Library Request (ELR) process that enables safe and efficient management of open source dependencies.
What You'll Learn
1
How to manage external libraries safely in software projects
2
Why reproducible builds are crucial for CI/CD pipelines
3
How to conduct legal and security checks on external libraries
Prerequisites & Requirements
- Basic understanding of CI/CD processes and dependency management
- Familiarity with build systems like Gradle and Maven(optional)
Key Questions Answered
What is the External Library Request (ELR) process?
The External Library Request (ELR) process at LinkedIn allows engineers to import and manage open source libraries safely. It involves requesting libraries, validating their metadata, checking legal compliance, and uploading them to an internal repository, thus ensuring a secure and reliable development environment.
How does LinkedIn handle external dependencies?
LinkedIn addresses external dependencies by importing them into their own repository through the ELR process. This allows for better auditing, security checks, and the ability to generate reproducible builds, reducing reliance on external repositories that may be unstable or insecure.
What are the challenges of managing external libraries?
Managing external libraries poses challenges such as increased repository size due to binary dependencies, lack of control over external repositories leading to potential build failures, and risks associated with buggy code or security vulnerabilities. The ELR process mitigates these issues by centralizing library management.
Why is it important to have reproducible builds?
Reproducible builds are essential as they ensure that the same software can be created from any point in time, reducing entropy in the build process. This is critical for maintaining consistency and reliability in software delivery, especially in a CI/CD environment.
Technologies & Tools
Build System
Gradle
Used as the primary build system for managing dependencies and building software products at LinkedIn.
Build System
Maven
Used for managing Java libraries and their dependencies.
Package Manager
Npm
Used for managing JavaScript libraries and their dependencies.
Dependency Management
Apache Ivy
Used for sharing code between projects and managing library metadata.
Key Actionable Insights
1Implement the External Library Request (ELR) process to streamline the management of open source libraries in your projects.By using the ELR process, teams can ensure that all dependencies are vetted for legal and security compliance, reducing risks associated with unverified libraries.
2Conduct regular audits of your external libraries to identify potential vulnerabilities and licensing issues.Regular audits help maintain a secure development environment and prevent legal complications that could arise from using non-compliant libraries.
3Utilize automated tools for dependency management to enhance the efficiency of your CI/CD pipeline.Automation reduces manual errors and speeds up the process of integrating external libraries, allowing teams to focus on development rather than administrative tasks.
Common Pitfalls
1
Failing to validate external libraries before importing them can lead to security vulnerabilities and legal issues.
Without proper validation, teams risk introducing unverified code into their projects, which can compromise the integrity and security of their software.
2
Over-relying on external repositories for dependencies can cause unexpected build failures.
If an external library is removed or compromised, it can disrupt the build process, highlighting the need for a robust internal management system like the ELR.
Related Concepts
Continuous Integration/Continuous Delivery (ci/Cd)
Dependency Management
Open Source Software Risks
Software Auditing