How We Built Fly Postgres

Fly.io is an ambivalent database provider—one might even use the word “reluctant”. The reasons for that are interesting, as is the way Fly Postgres works. When we relate this in conversations online, people are often surprised. So we thought we’d tak

Fly.io logo
Fly.io
•
Chris Nicoll, Shaun Davis
•15 min read•advanced•
--
•View Original
AWSCockroachDBConsulDockerGolangHAProxyHerokuMySQLPlanetScalePostgreSQLRedisSQLiteTimescaleDB

Overview

The article discusses the development of Fly Postgres, a database offering by Fly.io, detailing its evolution from a platform without durable storage to an automated database solution. It highlights the challenges faced, the technologies used, and the architectural decisions made to ensure reliability and performance.

What You'll Learn

1

How to deploy a two-node PostgreSQL cluster on Fly.io

2

Why using Stolon for high-availability PostgreSQL is beneficial

3

How to manage PostgreSQL instances with HAProxy

Prerequisites & Requirements

  • Understanding of PostgreSQL and its replication mechanisms
  • Familiarity with Docker and Fly.io platform(optional)

Key Questions Answered

How does Fly Postgres ensure high availability?
Fly Postgres achieves high availability through the use of Stolon, which manages leader election and failover processes. It utilizes a Consul cluster to track the state of PostgreSQL instances, allowing for automatic promotion of replicas to leader status in case of failure, ensuring minimal downtime.
What role does HAProxy play in Fly Postgres?
HAProxy acts as a load balancer for Fly Postgres, routing client requests to the current leader instance. It performs health checks on PostgreSQL nodes and ensures that only healthy instances receive traffic, which enhances reliability and performance.
What are the components of the Fly Postgres architecture?
The architecture of Fly Postgres includes several key components: Stolon keepers manage local PostgreSQL instances, sentinels monitor cluster state, and proxies route client connections. This setup allows for effective management and high availability of the database service.
What challenges did Fly.io face when building Fly Postgres?
Fly.io initially hesitated to offer a database service due to its focus on edge applications. Over time, they recognized the need for durable storage and automated database management, leading to the development of Fly Postgres and its unique architecture.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Database
Postgresql
Used as the primary database system for Fly Postgres.
Backend
Stolon
Manages high-availability configurations for PostgreSQL instances.
Backend
Haproxy
Acts as a load balancer for routing requests to PostgreSQL instances.
Infrastructure
Consul
Used for service discovery and managing the state of PostgreSQL clusters.

Key Actionable Insights

1
Implementing a high-availability PostgreSQL setup using Stolon can significantly reduce downtime for applications.
By utilizing Stolon, you can automate leader election and failover processes, ensuring that your database remains accessible even in the event of instance failures.
2
Using HAProxy in conjunction with PostgreSQL can improve connection management and load balancing.
HAProxy's ability to perform health checks and route traffic to healthy instances can enhance the performance and reliability of your database applications.
3
Regularly backing up your PostgreSQL database is crucial for data integrity and recovery.
Utilizing block-level snapshots can provide a reliable backup solution, allowing for quick recovery in case of data loss.

Common Pitfalls

1
Relying too heavily on Consul for cluster management can introduce additional failure points.
If Consul becomes unavailable, it can lead to outages in the PostgreSQL service. It's important to consider alternative solutions or reduce dependency on Consul for critical operations.

Related Concepts

High Availability In Databases
Postgresql Replication Strategies
Load Balancing Techniques
Service Discovery In Distributed Systems