Skip the API, Ship Your Database

My favorite part about building tools is discovering their unintended uses. It’s like starting to write a murder mystery book but you have no idea who the killer is! History is filled with examples of these accidental discoveries: WD-40 was original

Fly.io logo
Fly.io
•
Ben Johnson
•6 min read•advanced•
--
•View Original
GraphQLSQLSQLite

Overview

The article discusses the innovative approach of using LiteFS to eliminate the API layer between services by shipping SQLite databases directly to clients. It highlights the benefits and challenges of this method, including improved performance and flexibility in data access.

What You'll Learn

1

How to replace API layers with SQLite databases for internal services

2

Why using read-only database replicas can improve performance and reduce complexity

3

When to consider shipping databases instead of building APIs

Key Questions Answered

How can LiteFS be used to replace API layers between services?
LiteFS allows services to access a shared SQLite database directly, bypassing the need for an API layer. This approach simplifies data access and enables services to perform complex queries using SQL, improving performance and reducing the overhead associated with API calls.
What are the limitations of using read-only database replicas?
Read-only database replicas cannot be used for data mutations, which means clients still need an API for updates. Additionally, changes in the database structure can complicate client interactions, as they may not adapt automatically to schema changes without an API to manage those transformations.
What are the benefits of moving compute to the client with SQLite?
By pushing a read-only copy of the database to clients, they can utilize their own computing resources without affecting others. This allows for greater flexibility in query execution and eliminates concerns about resource contention among multiple tenants.
How does shipping a database affect data access restrictions?
Shipping a database can limit the ability to restrict data access, especially in multi-tenant environments. To mitigate this, using separate databases for each tenant can prevent unauthorized data access while maintaining the lightweight nature of SQLite databases.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Database
Litefs
Used to manage SQLite databases and provide read-only replicas to clients.
Database
Sqlite
Serves as the underlying database technology for storing and querying data.

Key Actionable Insights

1
Consider using LiteFS to eliminate the API layer for internal services, allowing direct access to SQLite databases.
This approach can significantly reduce the complexity of your architecture and improve performance, especially in scenarios where services require frequent data access.
2
Evaluate the trade-offs of using read-only database replicas in your applications.
While this method can enhance performance, it also introduces challenges in data mutability and access control that need to be carefully managed.
3
Utilize SQLite's advanced querying capabilities to empower clients with more flexible data access.
By allowing clients to run their own SQL queries, you can enable them to derive insights from data without the overhead of API calls.

Common Pitfalls

1
Assuming that shipping a database will eliminate all API needs.
While read-only replicas can simplify data access, clients will still require APIs for any data mutations, which can lead to confusion if not properly managed.
2
Neglecting the implications of database schema changes on client applications.
Without an API layer to abstract changes, clients may face issues if the underlying database structure changes, making it essential to plan for backward compatibility.

Related Concepts

Database Replication
API Design
SQL Querying
Multi-tenancy In Databases