Shopify’s Infrastructure Collaboration with Google

Dale Neufeld

We’re always working to deliver the best commerce experience to our merchants and their customers. We provide a seamless merchant experience while shaping the future of retail by building a platform that can handle the traffic of a Kylie Cosmetic flash sale (they sell out in 20 seconds), ship ne

Shopify

•

Dale Neufeld

•5 min read•intermediate•

--

•View Original

AWSDockerGoogle CloudKubernetesRails

Overview

Shopify is collaborating with Google to enhance its cloud infrastructure, focusing on scalability and performance for its commerce platform. The article outlines Shopify's journey to the cloud, including the adoption of Kubernetes and Docker, and highlights significant achievements in their migration process.

What You'll Learn

1

How to leverage Google Kubernetes Engine for container orchestration

2

Why migrating to Google Cloud can enhance scalability and performance

3

How to implement selective database migration with minimal downtime

Key Questions Answered

What are the key benefits of Shopify's collaboration with Google?

Shopify's collaboration with Google enhances scalability, security, and performance for its commerce platform. By leveraging Google's cloud infrastructure, Shopify can better serve its 600,000+ merchants and adapt to the growing demands of global commerce.

How has Shopify evolved its infrastructure over the years?

Shopify has evolved its infrastructure from a simple cloud solution to a robust system that includes database sharding, application sharding, and automated failovers. This evolution has allowed Shopify to handle increased traffic and improve service reliability as they scaled.

What specific tools has Shopify developed for its cloud migration?

Shopify developed the Shop Mover, a selective database migration tool that allows for rebalancing shops between database shards with an average downtime of just 2.5 seconds per shop. This tool is crucial for minimizing disruption during migration.

What percentage of Shopify's data center workloads have been migrated to Google Cloud?

Shopify has migrated over 50% of its data center workloads to Google Cloud, marking a significant step in their cloud transition strategy.

Key Statistics & Figures

Percentage of data center workloads migrated

50%

This applies to Shopify's ongoing migration efforts to Google Cloud.

Average downtime per shop during migration

2.5 seconds

This statistic reflects the efficiency of Shopify's Shop Mover tool.

Number of production services built

400+

This indicates the scale of Shopify's operations on their new platform as a service (PaaS

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Orchestration

Google Kubernetes Engine

Used for managing containerized applications at scale.

Containerization

Docker

Utilized for creating and managing containers to ensure predictable and repeatable builds.

Governance

Grafeas

An open-source initiative for auditing and governing the software supply chain.

Key Actionable Insights

1
Consider adopting container orchestration tools like Kubernetes to manage application scalability effectively.
As Shopify has demonstrated, Kubernetes provides a robust solution for managing containerized applications, allowing for better resource utilization and simplified deployment processes.

2
Utilize selective database migration tools to minimize downtime during infrastructure transitions.
Shopify's Shop Mover tool showcases how strategic planning and the right tools can significantly reduce the impact of migration on service availability.

3
Engage with open-source communities to stay updated on best practices and innovations.
Shopify's contributions to projects like Grafeas highlight the importance of collaboration in advancing technology and ensuring security in software supply chains.

Common Pitfalls

1

Failing to adequately plan for database migrations can lead to significant downtime.

Shopify's experience with the Shop Mover tool illustrates the importance of minimizing downtime during migrations to maintain service continuity.

Related Concepts

Cloud Infrastructure Management

Container Orchestration With Kubernetes

Database Sharding Techniques

Open-source Contributions In Software Development

18 minute read The mobile space has changed quickly, even within the past few years. At Shopify, the world’s largest Rails application, we have seen the growth and potential of the mobile market and set a goal of becoming a mobile-first company. Today, over 130,000 merchants are using Shopify Mobile to set up and run their stores from their smartphones. Through the inherent simplicity and flexibility of the mobile platform, many mobile-focused products have found success. This post was co-written with Arham Ahmed, and shout-outs to Sean Corcoran of MacStadium and Tim Lucas of Buildkite.

DockerJavaSwift

17 min read

Has Summary

--

NVIDIA

Advanced

Kubernetes For AI Hyperparameter Search Experiments

The software industry has recently seen a huge shift in how software deployments are done thanks to technologies such as containers and orchestrators.

DockerKubernetesAWS

20 min read

Includes Code

Has Summary

--

NVIDIA

Advanced

Fast and Scalable AI Model Deployment with NVIDIA Triton Inference Server

Deploying fast and scalable AI models with NVIDIA Triton Inference Server supports high-performance.

AWSDockerKubernetes

11 min read

Has Summary

--

These articles from Shopify and other leading engineering teams share similar topics with "Shopify’s Infrastructure Collaboration with Google". Explore more engineering insights on Docker, Java, Kubernetes.