Getting real with virtual threads
Overview
The article discusses the challenges and insights gained from deploying virtual threads in Java 21 at Netflix, particularly focusing on performance issues encountered with SpringBoot and Tomcat. It highlights the symptoms of application hang-ups and the diagnostic process undertaken to resolve these issues.
What You'll Learn
1
How to diagnose performance issues related to virtual threads in Java 21
2
Why understanding the blocking behavior of virtual threads is crucial for application performance
3
When to use jcmd for thread dumps instead of jstack for virtual threads
Prerequisites & Requirements
- Familiarity with Java concurrency concepts and virtual threads
- Experience with JVM diagnostics tools like jcmd and jstack(optional)
Key Questions Answered
What symptoms indicate that a Java application is hanging due to virtual threads?
Symptoms include a persistent increase in the number of sockets in closeWait state and instances that stop serving traffic while the JVM remains active. This indicates that the application is likely hanging due to blocking operations within virtual threads.
How do virtual threads interact with OS threads in Java 21?
Virtual threads are not mapped 1:1 to OS threads; they are scheduled on a fork-join thread pool. When a virtual thread blocks, it relinquishes the OS thread, allowing other virtual threads to use the same OS thread, which can lead to performance bottlenecks if all OS threads are pinned.
What issues arise from using synchronized blocks with virtual threads?
Using synchronized blocks can pin virtual threads to OS threads, preventing them from yielding and causing a situation where no OS threads are available for other virtual threads. This can lead to application hang-ups and increased closeWait sockets.
How can thread dumps help in diagnosing issues with virtual threads?
Thread dumps can reveal the state of virtual threads, including those that are blank and not running. However, since virtual threads do not appear in jstack dumps, using jcmd to obtain a complete view of virtual thread states is necessary for effective diagnosis.
Technologies & Tools
Backend
Java 21
The primary programming language used for microservices at Netflix, featuring virtual threads.
Backend
Springboot 3
Framework used for building microservices that encountered issues with virtual threads.
Backend
Tomcat
Web server that serves traffic on REST endpoints and was affected by the virtual thread implementation.
Key Actionable Insights
1Utilize jcmd for thread dumps when working with virtual threads to capture their state effectively.Since virtual threads do not appear in jstack-generated dumps, using jcmd is essential for diagnosing issues related to virtual threads in Java 21.
2Avoid using synchronized blocks with virtual threads to prevent them from being pinned to OS threads.This practice can lead to performance bottlenecks, as pinned threads block the execution of other virtual threads, resulting in application hang-ups.
3Monitor the number of sockets in closeWait state as an indicator of potential application hang-ups.A persistent increase in closeWait sockets can signal that the application is not processing requests effectively, often due to blocking operations within virtual threads.
Common Pitfalls
1
Failing to recognize that virtual threads can be pinned to OS threads when using synchronized blocks.
This can lead to performance issues as pinned threads block the execution of other virtual threads, causing the application to hang.
Related Concepts
Java Concurrency
Virtual Threads
Jvm Diagnostics
Performance Optimization