Visit the post for more.
Overview
The article features Mark Callaghan, a database engineer at Facebook, discussing the challenges and strategies involved in scaling MySQL on multi-core servers with fast storage. It covers topics such as innoDB compression, daily responsibilities, and the importance of curiosity and persistence in database engineering.
What You'll Learn
1
How to scale MySQL on multi-core servers with fast storage
2
Why innoDB compression is important for large databases
3
How to incrementally optimize MySQL performance
4
When to adapt database features for complex workloads
Prerequisites & Requirements
- Understanding of database management systems and MySQL
- Experience with performance testing and debugging(optional)
Key Questions Answered
How does your team scale MySQL on multi-core servers?
The team addresses scaling MySQL on multi-core servers by tackling fast-storage challenges that introduce performance bottlenecks. They aim to increase the number of evictions from 1,000 to 10,000 per second, focusing on incremental changes to identify and eliminate inefficiencies.
What challenges does innoDB compression present?
InnoDB compression could theoretically reduce database size by a factor of two, but adapting it for Facebook's complex, write-heavy workload presents challenges. The bottlenecks at serialization points are exacerbated due to increased workload, requiring careful adaptation of the feature.
What does a typical day look like for a database engineer at Facebook?
A typical day involves checking the health of the database, responding to emails, conducting code reviews, and running performance tests. Engineers spend half their time fixing MySQL stalls and the other half focused on performance improvements.
What makes a database engineer great?
A great database engineer embodies curiosity and persistence, similar to Scotty from Star Trek. They explore various systems to identify problems, ensuring they can effectively troubleshoot and optimize database performance.
Key Statistics & Figures
Evictions per second
10,000
The target number of evictions per second that the team aims to achieve, up from the previous limit of 1,000.
Potential database size reduction
2x
The theoretical reduction in database size achievable through innoDB compression.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Database
Mysql
Used as the primary database management system at Facebook.
Database
Innodb
A storage engine for MySQL that supports transactions and foreign keys, which the team is optimizing.
Key Actionable Insights
1Focus on incremental changes to optimize MySQL performance effectively.By making small adjustments and monitoring their impact, engineers can identify what works and what doesn't, leading to more reliable performance improvements.
2Adapt database features like innoDB compression for specific workloads.Understanding the unique demands of your workload allows for better utilization of database features, ensuring they contribute positively to performance rather than becoming bottlenecks.
3Prioritize making systems debuggable and predictable.This approach reduces risk and enhances the ability of operations teams to manage databases effectively, allowing for smoother operations and better sleep for on-call engineers.
Common Pitfalls
1
Failing to adapt database features to specific workloads can lead to performance issues.
When features like innoDB compression are not tailored to the demands of a write-heavy workload, they can create bottlenecks that hinder performance.
Related Concepts
Database Performance Optimization
Mysql Scaling Techniques
Innodb Storage Engine Features