How Pinterest Uses Python

This article discusses how Pinterest successfully reduced Android testing build times by over 36% through the implementation of a runtime-aware sharding mechanism.

The article reflects on a decade of AI platform development at Pinterest, detailing the evolution from fragmented machine learning stacks to a unified AI platform that supports various models.

The article discusses Pinterest's transition to Moka, a next-generation data processing platform built on AWS Elastic Kubernetes Service (EKS).

The article discusses how Pinterest scaled its machine learning infrastructure using Ray, extending its capabilities beyond training to include feature development, sampling, and labeling.

This article discusses how implementing the Lightning Memory-Mapped Database (LMDB) improved memory management and performance for Pinterest's API service.

The article discusses the implementation and operational benefits of the Unified PubSub Client (PSC) at Pinterest, highlighting improvements in developer velocity, stability, and scalability.

This article discusses the integration of Python-based anomaly detection algorithms into Pinterest's Warden platform, originally built in Java.

The article discusses how Pinterest improved its machine learning (ML) dataset iteration speed using Ray, an open-source framework for scaling AI and ML workloads.

The article discusses the development of a unified PubSub client library at Pinterest, aimed at improving the scalability, stability, and developer velocity of the Logging Platform.

The article discusses the critical issue of Python dependency confusion at Pinterest, highlighting the security risks associated with software supply chains.

The article outlines the engineering interview process at Pinterest, emphasizing a positive candidate experience and the importance of showcasing real-world problem-solving skills.

The article discusses how Pinterest leverages machine learning to foster a healthy comment ecosystem on its platform.

The article discusses SearchSage, a search query representation model developed by Pinterest to enhance the relevance of search results and user engagement.

The article discusses how Pinterest utilizes machine learning to maintain a positive comment ecosystem amidst a growing creator community.

The article discusses SearchSage, a search query representation developed at Pinterest to enhance search retrieval and ranking.

The article discusses how Pinterest utilizes Apache Spark SQL for interactive querying, detailing the architecture, challenges faced, and solutions implemented to enhance user experience.

The article discusses the importance of quality and testing of advertising metrics on mobile devices at Pinterest.

The article discusses Pinterest's efforts to develop fast and reliable iOS builds by optimizing their build process and adopting Bazel as their primary build tool.

Pinterest has open sourced Querybook, a collaborative big data hub designed to improve data access and analysis for teams, especially in a remote working environment.

The article discusses Guardian, a real-time analytics and rules engine developed by Pinterest's Trust & Safety team to combat spam.

The article discusses how Pinterest empowered its data scientists and machine learning engineers by building a PySpark infrastructure that addresses challenges faced with existing tools like Hive a...

The article discusses the concept of 'Pinterest Paths', which describes the exploration behavior of users on Pinterest as they navigate through related ideas.

The article discusses the importance of ad metrics quality and testing on mobile platforms at Pinterest.

The article discusses the open-sourcing of Singer, Pinterest's logging agent designed for efficient and reliable data collection from numerous hosts.

The article discusses how Pinterest improved the speed and reliability of its iOS builds by adopting Bazel as their build system.

The article discusses the interview process at Pinterest, emphasizing a positive candidate experience that reflects the company's values and culture.

The article discusses the importance of API profiling at Pinterest, focusing on optimizing the API Gateway service through memory profiling, operational cost calculations, and dead code detection.

The article discusses the technological framework that supports Pinterest's growth, detailing the company's mission to help users discover and do things they love.

The article discusses the challenges Pinterest faced with their Apache Thrift schemas, which had become tightly coupled and complex, leading to inefficiencies in development cycles.

Pinterest has announced the open-sourcing of ptracer, a syscall-tracing library for Python that enhances the efficiency and reliability of their large Python codebase.

The article discusses the challenges of managing Python code at scale across multiple repositories at Pinterest and introduces the solution of a Python monorepo called Python Commons.

The article discusses the implementation and functionality of the Pintrace Trace Analyzer, a tool developed by Pinterest for analyzing distributed trace data.

The article discusses how Pinterest improved its continuous integration (CI) process for iOS development by adopting Nix and Buildkite, moving away from Jenkins.

The article discusses the applications of Pintrace data at Pinterest, highlighting its role in improving backend service latency and debugging.

The article discusses Pinterest's implementation of distributed tracing using their open-source tool, Pintrace, which tracks requests across microservices in their backend.

The article discusses Pinterest's migration from a legacy template rendering engine to React, highlighting the performance improvements and developer efficiency gained through this transition.

The article discusses how Pinterest engineers leveraged existing tools from VOTE. org to encourage voter participation by embedding interactive iframes in Pins.

The article discusses the evolution of MySQL compression techniques at Pinterest, particularly focusing on the challenges faced with InnoDB page compression and the development of a new column comp...

The article discusses the challenges of replicability in data science, highlighting a case study at Pinterest where nine data scientists provided differing results to the same question.

The article discusses Pinterest's migration of its web framework from Backbone to React, focusing on the challenges and techniques used during the transition, particularly for Pinner profiles.

This article discusses the implementation details of Tracker, a system designed for ingesting MySQL data at scale, focusing on the improvements made to the data upload process to S3.

The article discusses Pinterest's approach to ingesting large volumes of MySQL data at scale through a system called Tracker.

The article discusses the implementation of a real-time metrics dashboard for monitoring A/B experiments at Pinterest.

The article discusses the development of Pinterest's A/B testing platform, highlighting the need for a reliable and user-friendly system to manage a large volume of experiments.

The article discusses Pinterest's internal web service designed to visualize unstructured data, helping to identify trends and user engagement patterns.

The article discusses the open-sourcing of KingPin, a toolset developed by Pinterest to enhance scalability and reliability in their infrastructure.

The article discusses Pinterest's approach to automating human evaluation for content relevance and safety through crowdsourcing.

The article introduces new open-source tools for the Elixir community, highlighting the advantages of Elixir and Erlang in handling concurrency and errors.

Skyline is an ETL-as-a-Service platform developed by Pinterest to streamline data processing and reporting for its users.

Pinterest has open-sourced Pinrepo, an artifact repository designed to efficiently store and serve build artifacts while addressing scalability and reliability challenges.