Systems @Scale Tel Aviv 2019 recap

Systems @Scale Tel Aviv is an invitation-only technical conference for engineers who build and maintain large-scale systems. Approximately 300 engineers from 70 companies gathered for the third Sys…

Meta logo
Meta
•
Tal Salmona
•7 min read•advanced•
--
•View Original
Java

Overview

The article recaps the Systems @Scale Tel Aviv 2019 conference, which brought together approximately 300 engineers from 70 companies to discuss the challenges and innovations in building and maintaining large-scale systems. Key topics included scaling data center infrastructure, mutation testing, data prefetching, and the development of a new ORM in Go.

What You'll Learn

1

How to implement mutation testing using the Chaos Monkey methodology

2

Why data prefetching is crucial for improving application responsiveness

3

How to manage trade-offs in data alignment across multiple sources

4

When to consider using a monorepo for large-scale projects

Key Questions Answered

What are the benefits of mutation testing in software development?
Mutation testing helps ensure the quality of unit tests by injecting bugs into the code to verify if the tests can detect them. This technique, now gaining traction due to faster CPUs and improved tools, allows developers to assess the effectiveness of their test suites and improve code reliability.
How does Facebook manage data prefetching to enhance user experience?
Facebook Lite uses data prefetching to create the illusion of instant data availability from local cache. However, excessive prefetching can lead to performance issues, so Facebook carefully balances data usage and resource management to optimize responsiveness without wasting resources.
What challenges arise from aligning data points at scale?
Aligning data from over 2,500 sources presents unique challenges, such as frequent changes in periodic data affecting real-time data. Singular addresses these issues by separating frequently and infrequently changed data and using consistent hashing to manage updates efficiently.
What innovations has Facebook made to handle monorepos effectively?
Facebook has extended its source control system to manage the complexities of a monorepo, which allows for faster development despite the challenges posed by a large codebase. Innovations include optimizing developer tools to maintain efficiency as the repository scales.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Testing Tool
Chaos Monkey
Used for mutation testing to ensure the robustness of unit tests.
Programming Language
Go
Main programming language for developing services in Facebook Connectivity.

Key Actionable Insights

1
Implement mutation testing to enhance the quality of your unit tests.
Mutation testing can reveal weaknesses in your test suite by ensuring that tests can catch introduced bugs. This practice is becoming more feasible with advancements in CPU speeds and testing tools.
2
Optimize data prefetching strategies to improve application responsiveness.
By carefully managing how much data is prefetched, you can enhance user experience without incurring performance penalties. This is particularly important in mobile applications like Facebook Lite.
3
Consider using a monorepo for large-scale projects to streamline code management.
Monorepos can simplify dependency management and improve collaboration among teams, but they require robust tools and practices to handle the associated challenges effectively.

Common Pitfalls

1
Over-prefetching data can lead to wasted resources and performance regressions.
It's essential to balance the amount of data prefetched to avoid unnecessary resource consumption, which can degrade application performance.

Related Concepts

Mutation Testing
Data Prefetching
Monorepo Management
Orm Development