Overview
The article discusses Uber's open-source tool, Denial By DNS, designed to prevent resource exhaustion caused by DNS outages. It details the technical background of DNS interactions, the impact of these outages on service availability, and the solutions implemented to mitigate such issues.
What You'll Learn
1
How to identify and mitigate DNS-related outages in web services
2
Why M:N scheduling can improve resource utilization in web applications
3
How to implement Denial By DNS to test for DNS vulnerabilities
Prerequisites & Requirements
- Understanding of DNS and its role in web services
- Familiarity with Node.js and asynchronous programming(optional)
Key Questions Answered
What caused the login service outage at Uber in 2016?
The outage was caused by an unresponsive third-party DNS server, which led to increased DNS response times and blocked the login service from reaching necessary internal and external services. This resulted in users being unable to log in to the Uber app.
How does Denial By DNS help prevent outages?
Denial By DNS is an open-source tool that tests whether a service's language is affected by DNS interactions that can lead to denial-of-service conditions. It allows developers to identify vulnerabilities and implement measures to mitigate them.
What are the different threading models discussed in the article?
The article discusses three threading models: the CGI model, which forks processes for each request; the thread pool model, which reuses a fixed number of threads; and the M:N model, which allows multiple threads to handle multiple requests simultaneously, optimizing resource usage.
What are some best practices to avoid DNS-related outages?
Best practices include avoiding synchronous DNS calls, replacing well-known domains with IP addresses, and using non-affected programming languages. These strategies help maintain service availability even during DNS failures.
Technologies & Tools
Some links below are affiliate links. We may earn a commission if you make a purchase.
Backend
Node.js
Used for building the login service that experienced the outage.
Networking
DNS
Critical for resolving domain names to IP addresses, affecting service availability.
Key Actionable Insights
1Implementing Denial By DNS can proactively identify potential vulnerabilities in your web service's DNS interactions.By using this tool, developers can simulate DNS outages and assess how their applications respond, allowing them to make necessary adjustments before issues arise in production.
2Consider using the M:N threading model for your applications to optimize CPU usage and reduce context switching.This model allows for better resource management by enabling threads to handle multiple requests simultaneously, which is particularly beneficial in high-traffic scenarios.
3Regularly test your applications against DNS failures to ensure resilience.By simulating DNS outages during development, teams can uncover weaknesses in their systems and implement robust error handling and fallback strategies.
Common Pitfalls
1
Relying on synchronous DNS calls can lead to service outages during DNS slowdowns.
When a service waits for a DNS response synchronously, it blocks the thread, preventing it from handling other requests, which can lead to resource exhaustion and downtime.
Related Concepts
DNS Interactions
Threading Models In Web Applications
Asynchronous Programming In Node.js