Carving The Scheduler Out Of Our Orchestrator

So, you want to build an app to rate sandwiches. Well, the world has a lot of different sandwiches. Pit beefs in Baltimore, Tonkatsu sandos in Shinjuku, and Cemitas in Puebla. You want real-time sandwich telemetry, no matter the longitude of the sand

Fly.io logo
Fly.io
•
Thomas Ptacek
•21 min read•advanced•
--
•View Original
AWSConsulDjangoDockerFlaskGolangJSONKubernetesNomadRailsRedisRustSinatraSQLite

Overview

The article discusses the evolution of Fly.io's scheduling system, detailing the transition from traditional orchestrators like Kubernetes and Nomad to a custom-built solution called flyd. It emphasizes the need for a more efficient and responsive scheduling mechanism tailored to the unique demands of their platform.

What You'll Learn

1

How to build a custom scheduling system for distributed applications

2

Why traditional orchestrators may not fit all use cases

3

How to implement a bidding-style resource allocation system

Prerequisites & Requirements

  • Understanding of distributed systems and orchestration concepts
  • Familiarity with Docker and virtual machine technologies(optional)

Key Questions Answered

What are the limitations of traditional orchestrators like Kubernetes?
Traditional orchestrators like Kubernetes can be overly complex and may not be suitable for all applications, especially those requiring rapid scaling and responsiveness. They often rely on centralized scheduling, which can lead to inefficiencies and delays in resource allocation.
How does Fly.io's custom scheduler, flyd, differ from Nomad?
Fly.io's flyd operates on a bidding-style resource allocation model, allowing for quick decision-making without the overhead of centralized consensus protocols. This contrasts with Nomad, which uses asynchronous scheduling and may introduce delays in job execution.
What is the role of flaps in Fly.io's architecture?
Flaps acts as a stateful proxy that interfaces with the flyd instances, facilitating communication and resource allocation across different regions. It helps manage the scheduling requests and ensures that resources are allocated efficiently.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Containerization
Docker
Used for running applications in isolated environments.
Orchestration
Nomad
Previously used orchestrator for managing application deployments.
Scheduling
Flyd
Custom-built scheduler for managing resource allocation in Fly.io's architecture.

Key Actionable Insights

1
Implement a bidding-style resource allocation system to enhance responsiveness in your applications.
This approach can significantly reduce the time taken to allocate resources, especially in environments where rapid scaling is necessary.
2
Consider the specific needs of your application when choosing an orchestrator.
Not all orchestrators are created equal; understanding the unique demands of your application can help you select the right tool for the job.
3
Leverage local state management for efficiency in distributed systems.
By allowing each worker to maintain its own state, you can reduce the complexity and overhead associated with centralized state management.

Common Pitfalls

1
Over-relying on centralized scheduling can lead to inefficiencies and delays.
This often happens when orchestrators are designed for large-scale environments but are applied to smaller, more agile applications.

Related Concepts

Distributed Systems
Resource Allocation Strategies
Custom Scheduling Algorithms