Updating Illustrations at Scale

The Polaris team creates tools, education and documentation that helps unite teams across Shopify to build better experiences. We created Polaris, our design system, and continue to maintain it. We are a multidisciplinary team with a range of experience. Some people have been at Shopify for over 6 years and others, like me, are a part of our Dev Degree program.

Shopify logo
Shopify
•
Emma Orhun
•8 min read•intermediate•
--
•View Original
esbuildGolangJavaScriptJSONNext.jsNode.jsPreact

Overview

The article discusses the challenges faced by the Polaris team at Shopify in updating illustrations across their product at scale. It details the development of an open-source tool, @shopify/get-repo-images, which streamlines the process of finding and managing illustrations across over 6000 repositories.

What You'll Learn

1

How to manage illustrations across multiple repositories efficiently

2

Why using Go can improve performance in large-scale applications

3

How to utilize open-source tools for image management in software projects

Prerequisites & Requirements

  • Understanding of GitHub repositories and image file formats
  • Familiarity with Node.js and Go programming languages(optional)

Key Questions Answered

What challenges did Shopify face in updating illustrations at scale?
Shopify faced fragmentation of illustrations across over 6000 repositories, making it difficult to track usage and updates. There were inconsistencies in file types and references, leading to inefficiencies in managing the illustrations.
How did the Polaris team improve the illustration update process?
The Polaris team developed @shopify/get-repo-images, an open-source tool that allows illustrators to find and manage images across multiple repositories efficiently. This tool enables sorting by file size and filtering by specific repositories, significantly streamlining the update process.
What programming languages were used in the development of the illustration management tool?
The tool was developed using Go for performance and Node.js for initial prototyping. The final implementation combined a Preact website with a simple API, enhancing the overall user experience.
What performance improvements were achieved with the new tool?
The new tool reduced the time to search and build the website from over 35 minutes to just 8 minutes, significantly enhancing the efficiency of managing illustrations across repositories.

Key Statistics & Figures

Time to search and build the website
8 minutes
This is a significant improvement from the previous time of over 35 minutes.
Number of repositories managed
over 6000
Illustrations were fragmented across this many repositories, complicating the update process.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Backend
Go
Used to build the @shopify/get-repo-images tool for performance.
Backend
Node.js
Initially used for prototyping the illustration management tool.
Frontend
Preact
Used to create the user interface for browsing and managing images.

Key Actionable Insights

1
Implement a centralized tool for managing visual assets across repositories to reduce fragmentation.
By utilizing a tool like @shopify/get-repo-images, teams can streamline the process of updating and managing images, ensuring consistency and saving time.
2
Consider using Go for performance-critical applications, especially when dealing with large-scale data processing.
Go's ability to handle parallel processing can significantly improve the performance of applications that require searching through millions of lines of code.
3
Encourage collaboration between developers and illustrators to ensure that design updates are effectively integrated into the codebase.
Regular communication can help identify gaps in the illustration update process and lead to more efficient workflows.

Common Pitfalls

1
Relying on outdated tools or methods for managing illustrations can lead to inefficiencies.
Without a centralized solution, teams may struggle to keep track of image usage and updates, leading to inconsistencies and wasted time.
2
Neglecting to consider performance when building applications that handle large datasets.
Using less efficient programming languages or methods can result in slow performance, as experienced with the initial Node.js implementation.

Related Concepts

Image Management Best Practices
Open-source Tool Development
Performance Optimization In Software Applications