How Uber Uses AWS

This article details how Uber built and scaled Apache Hudi to power one of the world's largest data lakes, managing 19,500 datasets with trillions of records across a multi-hundred-petabyte reposit...

This article discusses how Uber utilizes a pull-based ingestion model in OpenSearch™ to effectively index streaming data.

The article discusses the implementation of a Policy Simulator at Uber to enhance the safety and determinism of Identity and Access Management (IAM) policy changes.

The article discusses the evolution of Uber's Search Platform, highlighting its transition from Elasticsearch to an in-house solution called Sia, and ultimately to the adoption of OpenSearch.

The article discusses Uber's implementation of a configuration-driven archival and retrieval framework designed to manage vast amounts of regulatory data efficiently.

The article discusses how Uber utilizes Ray®, a general compute engine for Python®, to enhance the efficiency of its rides business through improved machine learning model performance and optimizat...

This article details Uber's migration of over a trillion entries of ledger data from DynamoDB to LedgerStore, focusing on the challenges, strategies, and outcomes of the process.

The article discusses how Uber's LedgerStore manages trillions of indexes to support its vast transactional data, emphasizing the architecture and indexing strategies that ensure data integrity and...

The article discusses Jupiter, Uber's new config-driven adtech batch ingestion platform, which replaces the legacy system MaRS.

DataCentral is Uber's proprietary platform designed for Big Data observability, chargeback, and governance.

The article discusses Uber's experience in adopting SPIFFE/SPIRE at scale, detailing the challenges and solutions encountered in implementing a Zero Trust security model across a complex microservi...

The article discusses how Uber utilizes Apache Pinot for real-time analytics of mobile app crashes, enhancing their ability to detect and resolve issues quickly.

The article discusses the evolution of Data Lifecycle Management (DLM) at Uber, detailing the journey from initial implementations to the development of a unified system.

The article discusses Uber's implementation of Attribute-Based Access Control (ABAC) to manage access across its microservices architecture.

Uber is committed to sustainability with a goal to become a zero-emission mobility platform by 2030 in North America and Europe, and by 2040 globally.

The article discusses Uber's implementation of security features for its Kafka infrastructure, detailing the importance of securing data integrity and access control.

This article details Uber's migration of financial data from DynamoDB to Docstore, highlighting the challenges faced and the architectural decisions made to ensure data integrity and operational ef...

This article discusses the development of Uber's Fulfillment Platform using Google Cloud Spanner, focusing on its architecture, scalability, and operational efficiency.

The article discusses how Uber's Global Scaled Solutions team transitioned from a traditional analytics architecture to a real-time analytics system using Redis, AWS Fargate, and the Dash framework.

The article discusses the integration of Elastic Horovod with Ray, focusing on how this combination enhances distributed deep learning training by enabling autoscaling and fault tolerance.

The article discusses Uber's development of uWorc, a no-code workflow orchestrator designed to simplify the creation of batch and streaming data pipelines.

Horovod v0. 21 introduces significant enhancements aimed at optimizing network utilization for distributed deep learning training.

The article introduces Athenadriver, an open-source database driver for Amazon Athena designed for Go, which facilitates seamless integration between Uber's business intelligence tools and AWS Athe...

The article discusses Uber's implementation of multi-tenancy within its microservice architecture, highlighting its benefits for stability, modularity, and developer velocity.

The article discusses how Uber manages its data workflows at scale, detailing the evolution from multiple overlapping data workflow systems to a centralized management system called Piper.

The article discusses the latest updates to Horovod, a distributed deep learning framework, which now includes support for PySpark and Apache MXNet, along with features aimed at enhancing training ...

The article introduces Alex Sergeev, the lead of the Horovod project at Uber, detailing the motivations behind open sourcing Horovod, a distributed deep learning framework.

Horovod, Uber's open-source distributed training framework, has joined the LF Deep Learning Foundation, enhancing its support for open-source innovation in AI and deep learning.

The article highlights seven open source projects showcased at the Uber Open Summit, emphasizing Uber's contributions to the open source community.

Uber's Big Data platform has evolved significantly, managing over 100 petabytes of data with minimal latency.

The article introduces Uber Engineering's Developer Experience (Dev Exp) team, which enhances the productivity of engineers by providing tools, documentation, and training.

The article discusses Uber's transition from IPv4 to IPv6, highlighting the necessity of this upgrade due to the rapid growth of its user base and the exhaustion of IPv4 addresses.

The article discusses Uber's development of a custom email Intrusion Detection System (IDS) to enhance security against phishing attacks.

The article discusses Pythagoras, a hyperfast artisanal pizza delivery service in San Francisco, and the challenges it faces in the on-demand economy.

The article discusses the use of the AWS IoT Button to simplify the process of requesting an Uber ride.