How Pinterest Uses PySpark

The article reflects on a decade of AI platform development at Pinterest, detailing the evolution from fragmented machine learning stacks to a unified AI platform that supports various models.

The article discusses Pinterest's transition to Moka, a next-generation data processing platform built on AWS Elastic Kubernetes Service (EKS).

This article discusses Pinterest's transition from a Hadoop-based platform to a Kubernetes-based data processing solution named Moka.

The article discusses the integration of Ray infrastructure at Pinterest, detailing the journey, challenges, and solutions implemented to optimize machine learning workflows.

The article discusses how Pinterest improved its machine learning (ML) dataset iteration speed using Ray, an open-source framework for scaling AI and ML workloads.

This article discusses Pinterest's implementation of a finer-grained access control (FGAC) framework to manage data access securely and efficiently within their data engineering platform.

This article discusses Pinterest's Batch Processing Platform, Monarch, focusing on efficient resource management to ensure quality of service (QoS) while maintaining cost efficiency.

The article discusses how Pinterest employs machine learning to combat spam and harmful content on its platform.

The article discusses how Pinterest employs machine learning to combat misinformation, hate speech, and self-harm content on its platform.

The article discusses how Pinterest empowered its data scientists and machine learning engineers by building a PySpark infrastructure that addresses challenges faced with existing tools like Hive a...