Introducing the Merge Queue

Shopify’s primary application is a monolithic Rails application that powers our cloud-based, multi-channel commerce platform for 600,000+ merchants in over 175 countries. As we continue to grow the number of developers working on the app, our tooling has grown with them.

Shopify logo
Shopify
•
Darren Worrall
•7 min read•intermediate•
--
•View Original
MonolithRails

Overview

The article introduces the Merge Queue, a feature developed by Shopify to enhance the merging process within their monolithic Rails application. It discusses the challenges faced with increasing developer activity and how the Merge Queue automates and streamlines pull request management, leading to safer and faster deployments.

What You'll Learn

1

How to implement a merge queue to automate pull request management

2

Why using a merge queue can improve deployment safety and speed

3

How to configure Shipit for optimal merge queue functionality

Prerequisites & Requirements

  • Understanding of trunk based development workflows
  • Familiarity with Shipit and GitHub integration(optional)

Key Questions Answered

How does the Merge Queue enhance the merging process at Shopify?
The Merge Queue automates the management of pull requests, allowing developers to enqueue their changes without leaving GitHub. This reduces the risk of merge conflicts and improves deployment speed by ensuring that only safe commits are merged when the master branch is in a stable state.
What challenges does Shopify face with its monolithic Rails application?
Shopify's growing number of developers leads to an increased volume of pull requests, which can cause merge conflicts and deployment delays. The need for better tooling to manage these merges became critical as the team expanded, prompting the development of the Merge Queue.
What features does Shipit provide for managing unsafe commits?
Shipit allows developers to mark commits as unsafe for deployment, preventing them from being deployed in isolation. It also features automatic revert detection, marking original commits as unsafe if a revert is detected, thus ensuring safer deployments.
How does the Merge Queue integrate with GitHub?
The Merge Queue is integrated into the Shipit tool, allowing developers to add pull requests to the queue directly within GitHub. This integration helps streamline the merging process and encourages developers to use the queue as the default method for merging changes.

Key Statistics & Figures

Percentage of pull requests using the merge queue
Over 90%
This statistic highlights the effectiveness and adoption of the Merge Queue within Shopify's core application.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Deployment Tool
Shipit
Used for managing deployments and integrating the merge queue functionality.
Version Control
Github
Integrated with Shipit to allow developers to manage pull requests and the merge queue directly.

Key Actionable Insights

1
Implementing a merge queue can significantly reduce merge conflicts and streamline the deployment process.
As development teams grow, the likelihood of merge conflicts increases. By automating the merge process, teams can focus on delivering value rather than managing conflicts.
2
Utilizing Shipit's features for marking unsafe commits can enhance deployment safety.
By automatically detecting reverts and marking related commits as unsafe, developers can avoid deploying potentially problematic changes, leading to a more stable production environment.
3
Encouraging the use of the merge queue as the default merging mechanism can improve team efficiency.
Adjusting the visibility of the merge buttons in GitHub can nudge developers towards using the merge queue, which has been shown to increase adoption rates.

Common Pitfalls

1
Developers may instinctively merge directly to master instead of using the merge queue.
This behavior can undermine the benefits of the merge queue, leading to increased merge conflicts and deployment delays. Training and adjustments in the UI can help mitigate this issue.

Related Concepts

Trunk Based Development
Deployment Automation
Continuous Integration Practices