Overview
The article announces the release of Voldemort 1.6.0, a distributed key-value storage system developed at LinkedIn. It highlights significant improvements, including new features, major changes, and enhancements aimed at increasing performance and usability.
What You'll Learn
1
How to implement zone-expansion capability in a distributed system
2
Why to use the Coordinator service and thin client for better communication with Voldemort
3
How to utilize new rebalance tools for enhanced cluster management
Prerequisites & Requirements
- Understanding of distributed systems and key-value storage concepts
- Familiarity with Java and REST APIs(optional)
Key Questions Answered
What are the new features introduced in Voldemort 1.6.0?
Voldemort 1.6.0 introduces several new features, including zone-expansion capability with abortable rebalance, new tools for zone-expansion, and the addition of a Coordinator service and thin client. These enhancements improve the system's scalability and communication efficiency.
How does the new zone-expansion feature work?
The zone-expansion feature allows for the addition of new zones in a distributed system, enabling efficient data management. It includes a Proxy-Put mechanism that ensures no data loss occurs during rebalancing, allowing operations to be safely aborted if necessary.
What major changes were made in this release?
Major changes in Voldemort 1.6.0 include the removal of donor-based rebalancing, an upgrade to BDB-JE 5 for improved performance, and enabling ZenStoreClient by default. These changes enhance the system's efficiency and reliability.
What improvements were made to the hinted handoff strategy?
The hinted handoff strategy has been rewritten to eliminate race conditions that caused write loss. The new approach synchronizes response processing in the client, ensuring better reliability when handling node failures.
Key Statistics & Figures
Operations per second
1 million
Voldemort achieved this milestone by serving operations across multiple zones and over 300 nodes.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Database
Bdb-je
Used for improved performance in handling large databases where the index does not fit in memory.
Programming Language
Java
Used for developing the new thin client that integrates with Voldemort.
Communication Protocol
REST API
Facilitates interactions between the thin client and the Coordinator service.
Key Actionable Insights
1Implement the new zone-expansion capability to enhance scalability in your distributed application.This feature allows for the addition of new zones without downtime, making it easier to manage growing data needs and improving overall system performance.
2Utilize the Coordinator service and thin client for more efficient interactions with Voldemort.These components streamline communication and reduce the overhead associated with the previous fat client, leading to improved application responsiveness.
3Adopt the new rebalance tools to maintain a balanced cluster topology.Tools like PartitionAnalysis and RebalancePlanner help identify and rectify imbalances in data distribution, ensuring optimal performance across the cluster.
Common Pitfalls
1
Failing to properly implement the new Proxy-Put mechanism during zone rebalancing can lead to data loss.
It's crucial to understand how this feature works to ensure that operations can be safely aborted without losing data during rebalancing.
2
Not upgrading to BDB-JE 5 could result in suboptimal performance for large databases.
Staying on outdated versions can hinder performance improvements and make it difficult to leverage new features effectively.
Related Concepts
Distributed Systems
Key-value Storage
Data Replication
Scalability In Distributed Architectures