Overview
The article discusses LinkedIn's approach to authorization within its extensive microservice architecture, emphasizing the importance of maintaining data security and trust. It outlines the challenges faced at scale, including managing access control lists (ACLs) and ensuring quick authorization checks.
What You'll Learn
1
How to implement an ACL-based authorization system in microservices
2
Why in-memory caching is critical for authorization checks
3
How to utilize Change Data Capture for cache invalidation
Prerequisites & Requirements
- Understanding of microservices architecture
- Familiarity with REST APIs and caching mechanisms(optional)
Key Questions Answered
How does LinkedIn ensure data security in its microservices?
LinkedIn ensures data security by implementing strict authorization controls through an ACL-based system. Each microservice can only access data based on defined business use cases, minimizing data exposure and potential damage from service compromises.
What challenges does LinkedIn face with ACLs at scale?
With over 700 microservices and tens of millions of calls per second, LinkedIn faces challenges in maintaining and updating ACLs efficiently. The need to quickly check authorizations, deliver changes, and manage a large number of ACLs is critical for operational success.
What technology does LinkedIn use for storing ACLs?
LinkedIn uses the Espresso database to store ACLs, which provides reliability and scaling capabilities. Additionally, a Couchbase cache is employed to enhance performance by serving most data from memory.
How does LinkedIn handle stale ACL data in the cache?
To manage stale ACL data, LinkedIn employs a Change Data Capture system using Brooklin, which notifies services when an ACL changes, allowing them to clear the cache and refresh their data accordingly.
Key Statistics & Figures
Number of microservices at LinkedIn
over 700
This extensive number of microservices contributes to the complexity of managing ACLs and ensuring secure data access.
Average rate of internal calls per second
tens of millions
This high volume necessitates efficient authorization checks and rapid ACL updates to maintain performance.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Database
Espresso
Used for storing ACLs with reliability and scaling capabilities.
Cache
Couchbase
Serves as a cache in front of Espresso to enhance performance by serving data from memory.
Data Pipeline
Brooklin
Used for Change Data Capture to notify services of ACL changes.
Message Queue
Kafka
Facilitates asynchronous, high-scale logging of authorization checks.
Framework
Rest.li
Used for publishing a REST API for managing ACLs.
Key Actionable Insights
1Implement in-memory caching for ACL data to reduce latency in authorization checks.By caching relevant ACL data in memory, services can make quick authorization decisions without incurring additional network calls, which is crucial for maintaining performance at scale.
2Utilize a Change Data Capture system to keep cache data fresh and relevant.Implementing a system like Brooklin can help ensure that any changes to ACLs are promptly reflected in the cache, preventing stale data issues that could lead to unauthorized access.
3Regularly review and optimize your ACL management processes.As the number of microservices and requests grows, it's essential to streamline ACL management to ensure that authorization remains efficient and secure.
Common Pitfalls
1
Failing to keep ACL data in memory can lead to significant latency in authorization checks.
If services must make network calls to retrieve ACL data for every request, it can severely degrade performance and scalability.
2
Overloading the ACL server with frequent updates can lead to increased load and potential bottlenecks.
It's important to balance the refresh rate of ACLs with the overall system load to maintain efficiency.
Related Concepts
Microservices Architecture
Access Control Lists
Data Security Practices