Overview
The article discusses Uber's development of a custom email Intrusion Detection System (IDS) to enhance security against phishing attacks. It outlines the operational benefits, architectural decisions, and the extensibility of the solution, emphasizing the importance of speed, reliability, and cost-effectiveness in their approach.
What You'll Learn
1
How to build a custom email IDS to enhance security against phishing attacks
2
Why using a cloud-based architecture can improve operational efficiency and cost management
3
How to leverage microservices for concurrent analysis in security systems
Prerequisites & Requirements
- Understanding of phishing attacks and email security
- Familiarity with AWS services like EC2, S3, and Lambda(optional)
Key Questions Answered
What are the main reasons Uber built their own email IDS?
Uber built their own email IDS to drive operational benefits in price, extensibility, and performance. They wanted full control over features and alerts to adapt to evolving threats in real time and to capture advanced insights for debugging intrusion alerts and email processing.
How does Uber ensure the speed and reliability of their email IDS?
Uber ensures speed and reliability by running two versions of their IDS: one in production and another for testing. This setup allows them to continuously develop and integrate new features without delaying email delivery, enabling alerts and deletions of malicious emails within a minute of detection.
What architectural strategies did Uber employ for their email IDS?
Uber's email IDS utilizes a microservice architecture that splits each email into pieces for parallel analysis through on-demand pipelines. This allows for flexibility and the integration of existing tools and services, enhancing security and performance.
How does Uber's email IDS handle cost management?
Uber's custom email IDS significantly reduces annual costs compared to commercial solutions by utilizing cloud-based services like AWS. They monitor costs closely and adjust based on needs, leveraging deduplication to minimize processing expenses.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Cloud Services
AWS
Used for hosting the email IDS and leveraging various services like EC2, S3, and Lambda.
Caching
Elasticache
Utilized for deduplication of processing events to speed up campaign processing.
Search And Analytics
Elasticsearch
Employed for clustering signals for campaign and impact analysis.
Key Actionable Insights
1Consider building a custom IDS if your organization faces significant phishing threats and existing solutions are insufficient.Custom solutions can provide tailored features and better adaptability to evolving threats, ultimately enhancing your security posture.
2Utilize cloud services for hosting security solutions to gain flexibility and cost efficiency.Cloud platforms like AWS can help manage resources dynamically, allowing for scaling based on demand and reducing operational costs.
3Implement a microservice architecture to facilitate concurrent analysis and development in security systems.This approach allows specialized teams to work on different components simultaneously, improving the overall effectiveness and speed of security measures.
Common Pitfalls
1
Relying solely on commercial IDS solutions can limit visibility into their operations.
Many commercial solutions act as black boxes, making it difficult to debug and understand their analysis processes. Building a custom solution can provide greater insight and control.
Related Concepts
Phishing Attacks And Their Impact On Enterprise Security
Cloud-based Security Solutions
Microservices Architecture In Security Systems