Spin Cycle: Shopify’s SFN Team Overcomes a Cloud-Development Spiral

The process of getting SFN working on Spin, Shopify’s cloud-development tool, is a great example of the kind of hidden difficulty in technical work that's more related to human systems.

Shopify logo
Shopify
•
Jen Spinney
•9 min read•intermediate•
--
•View Original
GraphQLKubernetesMonolithRails

Overview

The article discusses Shopify's transition to using Spin, a cloud-based development tool, to enhance the development process for the Shopify Fulfillment Network (SFN) team. It highlights the challenges faced during this transition, particularly regarding dependencies and human factors that impact software development.

What You'll Learn

1

How to set up a cloud-based development environment using Spin

2

Why managing dependencies is crucial for effective development in a cloud environment

3

How to identify and mitigate human factors affecting software development processes

Prerequisites & Requirements

  • Understanding of cloud development concepts
  • Familiarity with Kubernetes and SSH clients

Key Questions Answered

How does Shopify's SFN team utilize Spin for development?
The SFN team uses Spin to run development servers on-demand as containers in Kubernetes, allowing developers to work without local installations. This approach facilitates easy sharing of work in progress and reduces the risk of affecting long-lived databases during experiments.
What challenges did the SFN team face while transitioning to Spin?
The SFN team encountered issues with developers committing changes that broke functionality for Spin, highlighting the hidden difficulties in technical work related to human systems rather than just technical implementations.
What are the feedback loops affecting the use of Spin in SFN development?
Two feedback loops were identified: a negative Spiral of Struggle, where pain points deter developers from using Spin, and a positive Loop of Happy Equilibrium, where a good experience encourages continued use. The team aims to shift towards the positive loop.
How does SFN handle its dependencies in the Spin environment?
SFN manages dependencies by either running instances directly in Spin, mocking them, or using shared instances. This approach is essential for validating changes effectively, given the numerous dependencies SFN has.

Key Statistics & Figures

Number of SFN engineers deploying to production daily
150
This high frequency of deployments contributes to the dynamic nature of the SFN codebase, making it challenging to maintain stability during the transition to Spin.
Daily production deployments by SFN
30
With multiple deployments each day, the SFN team faces continuous changes, which complicates the integration of new tools like Spin.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Container Orchestration
Kubernetes
Used to run development servers on-demand as containers in the Spin environment.
Cloud Development Tool
Spin
Facilitates cloud-based development by allowing developers to work without local installations.

Key Actionable Insights

1
Focus on creating a minimal viable environment in Spin for common validation flows.
By prioritizing basic functionality, the SFN team can ensure that most developers can use Spin effectively, which is crucial for maintaining productivity during the transition.
2
Encourage developers to actively use Spin despite its limitations.
This practice helps identify pain points and gaps in functionality, allowing the team to address these issues collaboratively and improve the overall development experience.
3
Regularly update mocks for dependencies to prevent staleness.
As dependencies evolve, keeping mocks up-to-date is essential to ensure that development flows remain functional and relevant, thereby reducing the risk of regressions.

Common Pitfalls

1
Failing to manage dependencies effectively can lead to regressions in functionality.
As the SFN codebase is frequently updated, neglecting to ensure that all dependencies are compatible with Spin can result in broken features and increased frustration among developers.

Related Concepts

Cloud Development Practices
Dependency Management In Software Engineering
Human Factors In Software Development