The @Scale Conference is an invitation-only technical event for engineers who work on large-scale platforms and technologies. This year’s event took place on October 16 at the San Jose Convention C…
Overview
The 2019 @Scale Conference brought together over 1,300 engineers to discuss challenges and innovations in building scalable applications and services. Keynotes and sessions covered topics ranging from AI advancements to distributed databases and privacy in technology.
What You'll Learn
1
How to leverage self-supervised learning techniques for autonomous driving data
2
Why building respect into systems enhances user experience and trust
3
How to implement a global authorization system like Zanzibar
4
When to apply distributed SQL database principles in high-performance applications
Key Questions Answered
What are the main challenges of scaling AI systems?
Scaling AI systems involves addressing challenges like making AI work across diverse languages and accents, managing billions of personalization decisions, and ensuring responsible growth. These challenges require innovative approaches to learning and inference to effectively serve a global user base.
How does Zanzibar handle access control at scale?
Zanzibar is designed to manage trillions of access control lists and millions of authorization requests per second. It provides a uniform data model for expressing access control policies, ensuring external consistency and respecting causal ordering of user actions, which is crucial for maintaining privacy.
What techniques improve natural language understanding in production?
Techniques such as multinode training for complex models like BERT, and optimizing for speed and accuracy, are essential for effective natural language understanding in production. These methods help reduce training times significantly and enhance model performance across various applications.
What are the privacy challenges in DNS protocols?
Traditional DNS protocols expose user activity to third parties, leading to privacy risks. New mechanisms like DNS-over-TLS and DNS-over-HTTPS aim to encrypt DNS queries, but deploying these technologies involves technical and political challenges that need to be addressed.
Key Statistics & Figures
Number of attendees at @Scale Conference 2019
1,300
This attendance figure highlights the conference's significance in the engineering community.
Authorization requests handled by Zanzibar
millions per second
This showcases Zanzibar's capability to support large-scale applications effectively.
Access control lists managed by Zanzibar
trillions
This emphasizes the scale at which Zanzibar operates, catering to numerous Google services.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
Zanzibar
A global system for storing and evaluating access control lists.
Database
Dynamodb
A NoSQL database service designed for internet-scale applications.
Event Streaming
Kafka
A platform for building real-time data pipelines and streaming applications.
Machine Learning Framework
Pytorch
Used for accelerating research to production deployment in AI applications.
Key Actionable Insights
1Integrating self-supervised learning techniques can significantly enhance the performance of AI models in autonomous driving. By leveraging unlabeled data effectively, developers can improve perception and prediction capabilities without the extensive need for manual labeling.This approach is particularly useful in environments where data labeling is costly or impractical, allowing for more efficient use of available data.
2Building respect into systems is crucial for user trust and satisfaction. By considering user needs and vulnerabilities during the design phase, engineers can create products that are not only functional but also considerate of user privacy and security.This is especially relevant in applications dealing with sensitive user data, where a respectful approach can mitigate potential backlash and enhance user loyalty.
3Implementing a global authorization system like Zanzibar can streamline access control across various services. By providing a consistent model for access policies, organizations can ensure security and efficiency in user authorization processes.This is vital for large-scale applications where managing access control lists can become complex and unwieldy.
Common Pitfalls
1
Neglecting user privacy when logging data can lead to significant trust issues. Many applications log extensive user data to troubleshoot and improve services, but this can compromise user privacy if not handled correctly.
To avoid this, developers should implement privacy-preserving techniques, such as anonymizing data and using cryptographic methods to ensure that user identities are protected.
Related Concepts
AI/ML Advancements In Scaling Applications
Distributed Database Architectures
Privacy-preserving Technologies In Software Development