Introducing Roast: Structured AI Workflows Made Easy

Shopify’s Augmented Engineering DX team tackled developer productivity challenges like flaky tests and low test coverage using AI agents. They discovered that breaking complex tasks into discrete steps was key to reliable AI performance, leading to the creation and open-sourcing of Roast, a tool designed to structure AI workflows effectively.

Shopify logo
Shopify
•
Obie Fernandez
•10 min read•advanced•
--
•View Original
ClaudeGitHub ActionsJenkinsOpenAI APIRailsRubyYAML

Overview

The article introduces Roast, a structured AI workflow orchestration framework developed by Shopify to enhance developer productivity by integrating AI agents with traditional coding practices. It emphasizes the importance of structured workflows to improve reliability and efficiency in software development tasks.

What You'll Learn

1

How to create structured AI workflows using Roast

2

Why structured AI workflows improve developer productivity

3

When to use the CodingAgent for complex code modifications

4

How to leverage session replay for workflow development

Prerequisites & Requirements

  • Basic understanding of AI and workflow automation concepts
  • Ruby 3.0+ and an OpenAI API key
  • Optional: shadowenv and ripgrep for enhanced functionality(optional)

Key Questions Answered

What is Roast and how does it enhance developer productivity?
Roast is a workflow orchestration framework designed to integrate AI agents with traditional coding practices, providing a structured approach to improve reliability and efficiency in software development tasks. It allows developers to create reproducible and testable workflows that can interleave AI behavior with standard code execution.
How does the CodingAgent work within Roast?
The CodingAgent is integrated with Claude Code, allowing it to perform complex tasks such as bug fixing and performance optimization through iterative improvements. It operates within the structured workflows defined by Roast, providing a hybrid approach that combines AI capabilities with deterministic code.
What built-in tools does Roast provide for developers?
Roast includes several built-in tools such as ReadFile, WriteFile, UpdateFiles, Grep, and the CodingAgent. These tools facilitate various tasks like reading and writing files, applying patches, and executing shell commands, enhancing the overall workflow automation process.
What are some real-world applications of Roast at Shopify?
At Shopify, Roast has been used for various applications including analyzing test files for antipatterns, automating type safety with Sorbet, proactive site reliability monitoring, and gathering competitive intelligence. These use cases demonstrate its effectiveness in improving code quality and operational efficiency.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Backend
Ruby
Roast is implemented in Ruby and is distributed as a Ruby gem.
AI
Claude Code
Integrated with Roast to provide advanced AI capabilities through the CodingAgent.
AI
Raix
Provides an abstraction layer for AI interactions, enabling Roast to work seamlessly with different AI providers.

Key Actionable Insights

1
Implement structured workflows using Roast to enhance reliability in AI-assisted tasks.
By structuring workflows, developers can ensure that AI agents operate within defined parameters, reducing non-determinism and improving the overall reliability of automated processes.
2
Utilize the CodingAgent for complex problem-solving tasks that require iterative testing.
The CodingAgent's ability to adapt and refine solutions makes it ideal for tasks like bug fixing and performance optimization, where traditional deterministic approaches may fall short.
3
Take advantage of session replay features to streamline workflow development.
Session replay allows developers to save and resume workflows at any step, significantly reducing the need to rerun costly AI operations and speeding up the development cycle.

Common Pitfalls

1
Failing to structure AI workflows can lead to unreliable outcomes.
Without a structured approach, AI agents may produce inconsistent results, making it difficult to achieve reliable automation in development tasks.

Related Concepts

Ai-assisted Development
Workflow Automation
Ruby Programming
Software Engineering Best Practices