How Netflix Uses YAML

Netflix redesigned Maestro's internal workflow engine, replacing the legacy Conductor 2. x-based stateless worker model with a custom stateful actor model built on Java 21 virtual threads.

The article introduces the new Config feature in Metaflow, which allows users to configure all aspects of their flows, particularly decorators, prior to execution.

The article explores the Psyberg framework used by Netflix's Membership and Finance data engineering team, focusing on its two primary operational modes: stateless and stateful data processing.

The article discusses the implementation of sample data pipelines using Dataflow at Netflix, focusing on bootstrapping, standardization, and automation of batch data pipelines.

The article discusses Netflix Maestro, a next-generation workflow orchestrator designed to manage data and machine learning workflows at scale.

The article discusses the management of data pipeline assets at Netflix using a tool called Dataflow.

The article discusses Netflix's approach to enhancing security at scale through the development of Wall-E, a custom implementation of Zuul that integrates Single Sign-On (SSO) to streamline securit...

The article discusses the evolution of data movement at Netflix Studio through the implementation of a Data Mesh architecture.

The article discusses Bulldozer, a self-serve data platform developed by Netflix for efficiently moving batch data from data warehouse tables to online key-value stores.

The article announces Priam, a co-process designed to enhance the management and operational efficiency of Cassandra databases at Netflix.