Overview
The article discusses the process of upgrading LinkedIn's Pinot servers from Red Hat Enterprise Linux 6 (RHEL6) to RHEL7 with minimal service interruptions. It details the architecture of Pinot, the migration approach, and the results achieved post-upgrade, including significant performance improvements.
What You'll Learn
1
How to migrate servers with minimal downtime using a custom approach
2
Why upgrading to a newer OS can reduce CPU usage significantly
3
How to utilize rsync for efficient data transfer during migrations
Prerequisites & Requirements
- Understanding of server architecture and data migration strategies
- Familiarity with rsync and Linux command line(optional)
Key Questions Answered
What are the benefits of upgrading from RHEL6 to RHEL7?
Upgrading from RHEL6 to RHEL7 resulted in a 20 to 40% reduction in CPU usage, improved SSD performance, and enhanced automation capabilities on LinkedIn’s deployment platform. These benefits contribute to better resource management and operational efficiency.
How does Pinot handle server migrations?
Pinot manages server migrations by spinning up new machines, updating assignment policies, and performing parallel rsyncs from source to destination servers. This method allows for data staging and verification before switching traffic, minimizing downtime.
What challenges were faced during the RHEL7 upgrade?
The primary challenge was ensuring that data replication from the single source of truth to new servers did not degrade performance. The team developed a custom migration strategy to address this by staging data ahead of time to maintain operational capacity.
Key Statistics & Figures
CPU usage reduction
20 to 40%
Observed after upgrading from RHEL6 to RHEL7
Technologies & Tools
Operating System
Red Hat Enterprise Linux
Used for server management and performance improvements
Database
Pinot
Real-time analytics database utilized for member-facing features at LinkedIn
Tool
Rsync
Used for efficient data transfer during server migrations
Key Actionable Insights
1Implement a custom migration strategy that includes data staging to minimize downtime during upgrades.This approach allows for verification of new servers before they go live, ensuring that performance is not impacted during the transition.
2Consider the benefits of upgrading your operating system to improve resource efficiency and automation.The upgrade from RHEL6 to RHEL7 not only reduced CPU usage but also enhanced the overall performance of the Pinot servers, demonstrating the value of keeping systems updated.
Common Pitfalls
1
Failing to verify data replication before switching traffic to new servers can lead to performance issues.
This often happens when teams rush the migration process. Implementing a staged approach allows for thorough testing and validation.
Related Concepts
Server Migration Strategies
Operating System Upgrades
Data Replication Techniques