Overview
The article discusses Pinterest's transition from HBase, its first NoSQL datastore, to a new serving architecture with a unified storage service. It outlines the reasons for deprecating HBase, including high maintenance costs, missing functionalities, and increasing complexity, while also detailing the path to adopting TiDB as a replacement.
What You'll Learn
1
How to evaluate the need for migrating from legacy databases like HBase
2
Why high maintenance costs can signal the need for technology upgrades
3
When to consider adopting a NewSQL database like TiDB for scalability
Key Questions Answered
What were the main reasons for Pinterest to deprecate HBase?
Pinterest decided to deprecate HBase due to high maintenance costs, missing functionalities, increased system complexity, high infrastructure costs, and declining community support. These factors made it challenging to meet evolving business needs and maintain operational efficiency.
How did Pinterest manage its HBase deployment?
Pinterest hosted around 50 HBase clusters, utilizing approximately 9000 AWS EC2 instances and over 6 PBs of data. A primary-standby cluster setup was used for high availability, with write-ahead logs ensuring data consistency between clusters.
What alternative technology did Pinterest choose to replace HBase?
Pinterest adopted TiDB, a distributed NewSQL database, as a replacement for HBase. TiDB was selected for its scalability, powerful query capabilities, and support for ACID semantics, addressing the limitations faced with HBase.
What challenges did Pinterest face with HBase's functionalities?
HBase's limited functionalities, such as the lack of distributed transactions and global secondary indexing, led to bugs and inconsistencies, particularly impacting services like Zen. This made it difficult to meet evolving customer requirements.
Key Statistics & Figures
Number of HBase clusters at peak usage
50
Pinterest had around 50 clusters at its peak usage of HBase.
Number of AWS EC2 instances used
9000
The HBase deployment utilized approximately 9000 AWS EC2 instances.
Total data managed
over 6 PBs
Pinterest managed over 6 PBs of data within its HBase clusters.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Database
Hbase
Used as Pinterest's first NoSQL datastore, powering various systems.
Database
Tidb
Chosen as the replacement for HBase due to its scalability and support for ACID transactions.
Key Actionable Insights
1Evaluate the total cost of ownership for your database technologies regularly.High maintenance costs can indicate that a technology is becoming obsolete or inefficient. Regular evaluations can help identify when to consider migrating to newer solutions.
2Consider the scalability and functionality requirements of your applications before choosing a database.As business needs evolve, the limitations of a database can hinder growth. Choosing a technology that can adapt to changing requirements is crucial for long-term success.
3Engage with the community and industry trends to stay informed about database technologies.Waning community support for a technology can lead to challenges in finding expertise and resources. Staying connected with industry trends helps in making informed decisions about technology adoption.
Common Pitfalls
1
Underestimating the impact of tech debt on database performance and maintenance.
Legacy systems often accumulate tech debt, leading to increased costs and risks. Regular assessments and updates are essential to mitigate these issues.
2
Failing to adapt to evolving customer requirements.
As customer needs change, sticking with outdated technologies can result in service inconsistencies and bugs. It's crucial to choose technologies that can evolve with business demands.