Shadow Jobs

We take developer productivity pretty seriously at Slack. It’s multi-faceted, spreading across build speed, reliability, modernity, and more. One thing we really pride ourselves in is using the latest build tools when possible. Not necessarily bleeding edge (alphas, betas, RC, etc.), but the latest stable versions. Aside from the new features and improvements they bring,…

Slack logo
Slack
•
Zac Sweers
•9 min read•beginner•
--
•View Original
ChefEnvoyGitHub ActionsJavaKotlin

Overview

The article 'Shadow Jobs' discusses how Slack enhances developer productivity by continuously testing pre-release versions of essential build tools through custom Continuous Integration (CI) jobs called Shadow Jobs. This approach allows developers to catch regressions early and provide feedback to maintainers, ultimately leading to a smoother transition to stable tool versions.

What You'll Learn

1

How to design and implement Shadow Jobs in your CI workflow

2

Why testing pre-release versions of build tools can improve software stability

3

When to utilize matrix builds for testing multiple configurations

Prerequisites & Requirements

  • Basic understanding of Continuous Integration and GitHub Actions
  • Familiarity with Gradle and Kotlin(optional)

Key Questions Answered

How does Slack implement Shadow Jobs in their CI process?
Slack implements Shadow Jobs by creating custom CI workflows in GitHub Actions that test pre-release versions of build tools against their codebase. This allows them to catch regressions early and provide timely feedback to maintainers, ensuring a smoother transition to stable releases.
What are the benefits of using Shadow Jobs for developers?
Using Shadow Jobs allows developers to test new versions of build tools continuously, which helps identify potential issues early. This proactive approach not only increases confidence in updating dependencies but also provides valuable feedback to tool maintainers, enhancing overall software quality.
What is the significance of matrix builds in testing configurations?
Matrix builds enable testing of multiple configurations simultaneously, allowing Slack to verify the compatibility of different Java versions, AGP versions, and Kotlin IR settings. This approach reduces redundant builds and ensures comprehensive coverage of potential issues across various setups.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

CI/CD
Github Actions
Used for creating custom CI workflows to implement Shadow Jobs.
Build Tool
Gradle
Used for building and testing the codebase in conjunction with Shadow Jobs.
Programming Language
Kotlin
The primary language used in the codebase being tested with Shadow Jobs.

Key Actionable Insights

1
Implement Shadow Jobs in your CI pipeline to continuously test pre-release versions of your build tools.
This practice helps catch regressions early and provides timely feedback to maintainers, ultimately leading to more stable software releases.
2
Utilize matrix builds to test various configurations in parallel, enhancing the efficiency of your CI process.
By testing multiple combinations of dependencies and settings, you can identify compatibility issues faster and reduce the risk of introducing bugs.

Common Pitfalls

1
Failing to test new versions of build tools until late in the development cycle can lead to significant regressions.
By adopting Shadow Jobs, teams can avoid this pitfall by continuously testing new tools, ensuring that issues are identified and addressed early.

Related Concepts

Continuous Integration
Github Actions
Gradle
Kotlin