LiteFS Cloud: Distributed SQLite with Managed Backups

We love SQLite in production, and we’re all about running apps close to users. That’s why we created LiteFS: an open source distributed SQLite database that lives on the same filesystem as your application, and replicates data to all the nodes in you

Darla Magdalene Shockley, Ben Johnson
8 min readintermediate
--
View Original

Overview

LiteFS Cloud is a new service from Fly.io that provides managed backups and point-in-time restores for LiteFS, a distributed SQLite database. It allows users to easily back up their databases with high frequency and restore them to any point in time within the last 30 days.

What You'll Learn

1

How to set up LiteFS Cloud for managed backups

2

Why LiteFS Cloud is beneficial for disaster recovery

3

How to restore your database to any point in time with 5-minute granularity

Prerequisites & Requirements

  • Basic understanding of SQLite and distributed databases
  • Access to Fly.io dashboard

Key Questions Answered

How does LiteFS Cloud manage backups for SQLite databases?
LiteFS Cloud provides managed backups by creating snapshots every five minutes, allowing users to restore their databases to any point in time within the last 30 days. This service eliminates the need for users to manually manage backups, making it easier to ensure data reliability.
What is the purpose of the Lite Transaction File (LTX) in LiteFS?
The Lite Transaction File (LTX) is designed for fast and flexible replication and recovery in LiteFS. It represents a change set of database pages, allowing for efficient storage and management of transactions, which is crucial for maintaining data integrity in distributed environments.
What are the compaction levels in LiteFS and how do they work?
LiteFS uses multi-level compactions to optimize data storage. It includes daily snapshots, hourly compactions, and five-minute interval files, which allow for efficient restoration by replaying only the necessary changes to reach a specific timestamp.
What are the future plans for LiteFS Cloud?
Future plans for LiteFS Cloud include features such as uploading databases directly through the Fly.io dashboard, downloading point-in-time snapshots, and cloning LiteFS Cloud clusters for staging environments. These enhancements aim to improve usability and support for various deployment scenarios.

Key Statistics & Figures

Snapshots per month
8760 snapshots
LiteFS Cloud creates a snapshot every five minutes, totaling 8760 snapshots in a month.

Technologies & Tools

Database
Litefs
LiteFS is used as a distributed SQLite database that enables local reads and data replication across application nodes.
Storage
S3
S3 is used for high-durability, high-availability object storage for LiteFS Cloud backups.

Key Actionable Insights

1
Implement LiteFS Cloud to automate your backup processes and enhance disaster recovery capabilities.
By using LiteFS Cloud, you can ensure that your SQLite databases are backed up every five minutes, significantly reducing the risk of data loss and simplifying recovery processes.
2
Utilize the compaction features of LiteFS to optimize your database performance.
Understanding how LTX files and compactions work can help you manage your database more efficiently, reducing storage costs and improving read performance.
3
Take advantage of the near-instantaneous restore feature to minimize downtime during incidents.
With the ability to restore to any point in time within the last 30 days, you can quickly recover from data corruption or loss, ensuring business continuity.

Common Pitfalls

1
Failing to implement a proper backup strategy can lead to significant data loss.
Without automated backups like those provided by LiteFS Cloud, you risk losing critical data due to unforeseen incidents. Regularly scheduled backups are essential for maintaining data integrity.

Related Concepts

Distributed Databases
Sqlite
Backup And Recovery Strategies
Data Replication