Scalable data classification for security and privacy

Nikolay Pavlovich Laptev

What the research is: We’ve built a data classification system that uses multiple data signals, a scalable system architecture, and machine learning to detect semantic types within Facebook at scal…

Overview

The article discusses a scalable data classification system developed at Facebook that utilizes machine learning and multiple data signals to enhance security and privacy. It highlights the system's ability to classify both persisted and nonpersisted data, enabling organizations to enforce privacy and security policies effectively.

What You'll Learn

1

How to implement a scalable data classification system using machine learning

2

Why traditional data loss prevention systems are insufficient for large organizations

3

When to use Bernoulli sampling for data scanning in classification tasks

Key Questions Answered

How does the data classification system at Facebook work?

The system classifies data by learning about data assets through metadata collection and scanning. It employs Bernoulli sampling for persisted data and provides real-time predictions for nonpersisted traffic via an online API, ensuring effective policy enforcement.

What are the advantages of using machine learning in data classification?

Machine learning improves classification accuracy, especially for unstructured data, compared to manual heuristics. The modularized PyTorch-based model allows for quick updates without degrading overall performance, making it adaptable to changing data types.

What types of data does the classification system handle?

The classification system handles both persisted data stored in offline data stores and nonpersisted data, such as real-time traffic, ensuring comprehensive coverage for privacy and security policies across various formats.

Why is visibility into data metadata important for organizations?

Visibility into data metadata allows organizations to enforce privacy policies effectively, make informed decisions quickly, and reduce errors by providing necessary information for data handling and classification.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Machine Learning

Pytorch

Used for building the machine learning model that classifies unstructured data.

Key Actionable Insights

1
Implement a data classification system that leverages machine learning for better accuracy in identifying sensitive data types.
This approach allows organizations to adapt to evolving data landscapes and ensures compliance with privacy regulations by automating the classification process.

2
Utilize Bernoulli sampling to efficiently scan large datasets without overwhelming system resources.
This technique enables organizations to balance performance and resource usage, making it feasible to classify vast amounts of data in real-time.

3
Consider integrating real-time classification APIs for nonpersisted data to enhance data security measures.
Real-time classification helps in monitoring and protecting sensitive information as it flows through the organization, reducing the risk of data breaches.

Common Pitfalls

1

Relying solely on manual heuristics for data classification can lead to inaccurate results, especially with unstructured data.

This happens because manual methods often fail to capture the complexity and variability of unstructured data, making it essential to incorporate machine learning for improved accuracy.