How Meta Uses Apache Spark
5 engineering articles about Apache Spark from Meta's engineering team
Other Meta Technologies
Other Companies Using Apache Spark
Articles
Filter:
The article discusses the collaboration between Meta, Voltron Data, and the Apache Arrow community to align Apache Arrow with Velox, Meta's open-source execution engine.
Pedro Pedreira
10 min read
Has Summary
--
Horizon: The first open source reinforcement learning platform for large-scale products and services
Horizon is the first open source end-to-end platform that employs applied reinforcement learning (RL) to optimize systems in large-scale production environments.
Jason Gauci
12 min read
Has Summary
--
The article discusses how Facebook utilized Apache Spark for large-scale language model training, highlighting the transition from a Hive-based solution to a Spark-based pipeline.
Tejas Patil
13 min read
Includes Code
Has Summary
--
This article provides a comprehensive comparison of two state-of-the-art graph processing systems, Apache Giraph and GraphX, focusing on their performance, scalability, and usability for large-scal...
Maja Kabiljo
20 min read
Includes Code
Has Summary
--
The article discusses Facebook's experience in scaling Apache Spark to handle a 60 TB+ production use case, focusing on the migration from a Hive-based pipeline to a more efficient Spark implementa...
You've reached the end! All 5 articles loaded.