How Uber Uses SQL

This article discusses the improvements made to MySQL cluster uptime at Uber through the implementation of MySQL Group Replication (MGR).

The article discusses Uber's implementation of I/O observability for its massive petabyte-scale data lake, focusing on the challenges and solutions in monitoring data access patterns across its hyb...

This article discusses the rebuilding of Uber's Apache Pinot™ query architecture, focusing on the transition from Neutrino to a new query system that utilizes Pinot's Multi-Stage Engine Lite Mode.

Uber's migration from Spark 2. 4 to Spark 3. 3 involved upgrading over 2 million Spark applications, utilizing innovative automation tools like Iron Dome.

The article discusses Finch, Uber's conversational AI data agent designed to streamline financial data retrieval within the Slack environment.

The article discusses the evolution of Uber's Search Platform, highlighting its transition from Elasticsearch to an in-house solution called Sia, and ultimately to the adoption of OpenSearch.

This article details Uber's migration from Apache Hive to Apache Spark SQL for ETL workloads, highlighting the motivations behind the transition, the architecture involved, and the challenges faced...

The article discusses the MySQL fleet at Uber, which consists of over 2,300 independent clusters that support critical operations for the platform.

The article discusses how Uber leverages Neutrino, an internal fork of Presto, to efficiently serve millions of queries to Apache Pinot, a real-time OLAP database.

The article discusses Uber's implementation of Presto Express, an enhancement to the Presto SQL query engine aimed at reducing the end-to-end Service Level Agreement (SLA) for short-running queries.

The article discusses how Uber optimizes the training of Large Language Models (LLMs) using both open-source and in-house models.

The article discusses Preon, a microservice developed by Uber for intelligent and efficient query analysis using the Presto SQL engine.

The article discusses QueryGPT, a tool developed by Uber that converts natural language prompts into SQL queries using generative AI.

The article discusses the Sparkle framework developed by Uber to standardize modular ETL processes, enhancing developer productivity and data quality.

The article discusses Uber's upgrade of its MySQL fleet from version 5. 7 to 8. 0, detailing the motivations, challenges, and solutions implemented during the process.

Uber is modernizing its batch data infrastructure by migrating to Google Cloud Platform (GCP) to enhance data analytics and machine learning capabilities.

The article discusses Uber's journey in enhancing its Palette Meta Store, focusing on the challenges faced, the solutions implemented, and the resulting improvements in machine learning feature man...

The article discusses the Cinnamon Auto-Tuner, a system designed to adaptively manage concurrency in production environments.

The article discusses CheckEnv, a tool developed by Uber for fast detection of remote procedure calls (RPCs) between different environments using graph technology.

This article discusses Uber's experience migrating a large-scale invoice generation service from a legacy system to a new service called Invoicer.

The article discusses the evolution of Data Lifecycle Management (DLM) at Uber, detailing the journey from initial implementations to the development of a unified system.

The article discusses Spark Analysers, a system developed by Uber to identify anti-patterns in Spark applications.

The article discusses how Uber implemented an incremental ETL process using Apache Hudi to manage its transactional data lake.

The article discusses D3, an automated system developed by Uber to detect data drifts in datasets, which is crucial for maintaining data quality and ensuring the performance of machine learning mod...

The article discusses how Uber reduced its logging costs significantly by integrating the Compressed Log Processor (CLP) into its logging architecture.

This article discusses Uber's migration from MySQL to MyRocks, a storage engine that integrates with RocksDB, to address disk space bottlenecks and improve operational efficiency.

The article discusses how Uber integrates Presto® and Apache Kafka® to enhance its big data analytics capabilities.

This article discusses the implementation of finer-grained encryption in Apache Parquet™, focusing on how it addresses data access restrictions, retention, and encryption at rest.

The article discusses Project RADAR, an intelligent fraud detection system developed by Uber that integrates machine learning and human expertise to identify and mitigate fraudulent activities in r...

This article discusses the development of Uber's Fulfillment Platform using Google Cloud Spanner, focusing on its architecture, scalability, and operational efficiency.

This article discusses Uber's implementation of a real-time exactly-once ad event processing system using open-source technologies such as Apache Flink, Kafka, and Pinot.

The article discusses Uber's initiatives to enhance the efficiency of its Big Data platform, focusing on cost reduction through optimizations in file formats, HDFS erasure coding, YARN scheduling i...

Uber’s Finance Computation Platform (FCP) is designed to handle the scale and complexity of financial transactions across its various services.

The article discusses Uber's comprehensive re-architecture of its Fulfillment Platform, aimed at enhancing its Go/Get strategy.

The article discusses Uber's 'Orders Near You' feature, which utilizes real-time geospatial data analytics to enhance user experience in the Uber Eats app.

The article discusses the development and implementation of Charon, a real-time analytics framework at Uber designed for automating merchant live monitoring.

The article discusses Uber's journey towards establishing a better data culture by addressing critical data issues and implementing a holistic approach to data management.

The article discusses the application of machine learning in internal auditing, specifically focusing on the challenges and methodologies used at Uber to analyze sparsely labeled data.

The article discusses the evolution of Uber's Schemaless datastore into a distributed SQL database called Docstore, highlighting its features, architecture, and motivation behind the transition.

Uber has developed a centralized, schema-agnostic log analytics platform that enhances logging efficiency and reliability.

The article discusses Uber's journey towards metric standardization through the development of uMetric, a unified internal metric platform.

The article discusses Uber's development of uWorc, a no-code workflow orchestrator designed to simplify the creation of batch and streaming data pipelines.

The article discusses Uber's Databook, an in-house platform designed to manage and surface metadata related to various data entities.

The article discusses Uber's experience operating Apache Pinot at scale, detailing its role in enabling real-time analytics across various use cases.

The article introduces Athenadriver, an open-source database driver for Amazon Athena designed for Go, which facilitates seamless integration between Uber's business intelligence tools and AWS Athe...

This article discusses the optimization of JVM memory and garbage collection (GC) for large-scale services at Uber, focusing on the challenges and solutions implemented to enhance performance and r...

The article discusses how Uber engineered SQL support on Apache Pinot, enhancing real-time analytics capabilities for their Big Data stack.

The article discusses Uber's open source initiatives in 2019, highlighting the company's contributions to the open source community, the establishment of the Open Source Program Office (OSPO), and ...

The article discusses Uber's advancements in data infrastructure during 2019, focusing on how data science was leveraged to optimize performance and manage vast amounts of data.

The article discusses the evolution of the Michelangelo model representation at Uber to enhance flexibility and scalability in machine learning model serving.