Visit the post for more.
Overview
Beringei is a high-performance time series storage engine developed by Facebook to address the limitations of their previous HBase-backed time series database. It is optimized for real-time health and performance monitoring, allowing for fast data storage, querying, and analysis.
What You'll Learn
1
How to implement a high-performance time series database for real-time monitoring
2
Why data compression is critical for efficient time series storage
3
When to use Beringei as an embedded library versus a standalone service
Key Questions Answered
What are the performance improvements of Beringei compared to HBase?
Beringei significantly improves performance with a read latency of approximately 65 microseconds for read requests and a write availability of data within 300 microseconds after a put request. This is several orders of magnitude faster than the previous HBase system.
How does Beringei achieve high data compression rates?
Beringei uses a lossless streaming compression algorithm that compresses time series data by at least 90 percent. It employs delta-of-delta encoding for timestamps and an XOR-based method for values, optimizing storage efficiency.
What are the primary use cases for Beringei?
Beringei can be used as a simple, sharded service for time series query requests or as an embedded library for efficient time series data storage. This flexibility allows it to serve various performance monitoring needs.
What is the maximum number of time series Beringei can handle?
Beringei currently stores up to 10 billion unique time series and serves 18 million queries per minute, making it capable of handling large-scale performance and health monitoring at Facebook.
Key Statistics & Figures
Read latency
65 microseconds
This is the p95 server response time for read requests in Beringei.
Write availability
300 microseconds
This is the delay between a counter being written to Beringei and being available for consumption.
Unique time series stored
10 billion
Beringei currently manages this volume of unique time series data.
Queries served per minute
18 million
This is the query load that Beringei handles at Facebook.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Database
Beringei
High-performance time series storage engine for monitoring systems.
Visualization
Grafana
Integration for visualizing time series data stored in Beringei.
Key Actionable Insights
1Implement Beringei as an in-memory time series database to enhance the performance of monitoring systems.Using Beringei allows for real-time data access and analysis, which is crucial for identifying and diagnosing system issues quickly.
2Utilize Beringei's streaming compression capabilities to reduce storage overhead significantly.By achieving over 90 percent compression, Beringei minimizes the memory footprint of time series data, allowing for more efficient resource utilization.
3Consider using Beringei as an embedded library for custom performance monitoring solutions.This approach can provide a high-performance storage backend similar to RocksDB, tailored to specific application needs.
Common Pitfalls
1
Assuming that existing disk-based solutions can handle the scalability needs of modern monitoring systems.
Many traditional databases are not optimized for real-time analysis, leading to slow query performance and timeouts, as experienced with the previous HBase system.
2
Neglecting the importance of data compression in time series databases.
Without effective compression, storage overhead can become unmanageable, impacting performance and resource allocation.