Caching Without Marshal Part 1: Marshal from the Inside Out

Shopify wanted a cache format that would not blow up when we shipped code changes. Part one of Caching Without Marshal describes Marshal, Ruby’s ultimate sharp knife, able to transform almost any object into a binary blob and back.

Shopify logo
Shopify
•
Chris Salzberg
•12 min read•advanced•
--
•View Original
ActiveRecordCachingMemcachedMessagePackMonolithRailsRedisRuby

Overview

The article discusses the critical role of caching in Rails applications and the inherent risks associated with using Ruby's Marshal for serialization. It highlights a past incident at Shopify that underscored the dangers of relying on Marshal and introduces MessagePack as a safer alternative for serialization in caching.

What You'll Learn

1

How to identify the risks associated with using Marshal for caching in Rails applications

2

Why using MessagePack can enhance cache safety and efficiency

3

When to consider alternative serialization formats for caching

Prerequisites & Requirements

  • Understanding of caching concepts and Rails framework
  • Familiarity with Ruby programming(optional)

Key Questions Answered

What are the risks of using Marshal for caching in Rails?
Using Marshal for caching can lead to issues when code changes, as it serializes class information alongside object data. This can cause cache collisions when new code introduces classes that old code does not recognize, leading to exceptions and application incidents.
How does MessagePack improve upon Marshal for caching?
MessagePack is a binary serialization format that is more compact and has stricter typing than Marshal. It avoids the pitfalls of caching arbitrary objects, making it a safer choice for serialization in Rails applications, particularly at scale.
What incident at Shopify highlighted the dangers of Marshal?
A developer's refactor of beta flags in Shopify's core monolith led to a flood of exceptions in production due to cached objects serialized with old class information. This incident underscored the risks of using Marshal for caching.
What changes were made in Rails 7 regarding caching?
Rails 7 introduced improvements in cache space allocation by optimizing the serialization of cached objects. However, the fundamental issues with using Marshal as a serialization format remain unchanged.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Serialization
Messagepack
Used as an alternative to Marshal for safer caching in Rails applications.
Framework
Rails
The framework within which caching and serialization issues are discussed.

Key Actionable Insights

1
Evaluate your current caching strategy to identify potential risks associated with using Marshal.
Understanding the limitations of Marshal can help you prevent future incidents related to cache collisions, especially when deploying code changes.
2
Consider migrating to MessagePack for serialization in caching to enhance safety and efficiency.
Switching to MessagePack can mitigate the risks associated with Marshal and provide a more robust solution for handling complex data structures in Rails applications.
3
Implement stricter guidelines on what types of objects can be cached in your Rails applications.
By limiting the types of objects stored in the cache, you can reduce the likelihood of encountering issues during code deployments.

Common Pitfalls

1
Assuming that all objects are safe to cache without understanding the serialization format used.
This can lead to cache collisions and exceptions when code changes introduce new classes that are not recognized by previously cached objects.
2
Neglecting to review the implications of using Marshal for caching in a production environment.
Failing to recognize the risks associated with Marshal can result in significant downtime and errors during deployment.