Using Java Virtuals Threads for Asynchronous Programming
Virtual Threads is a feature in Java 21. The key purpose of this feature is to maximize utilization of idle threads. As per Java documentation using the virtual threads will yield the maximum results when the processing application is low on CPU usage and high on I/O, network calls. Each Virtual thread is associated with a platform thread (also called carrier thread) which is a JVM wrapper around a OS thread.
A virtual thread associated to a task mounts on a platform thread to perform some processing. When an I/O call or network call is done and the virtual thread is idle waiting for the response the platform thread is disassociated from the virtual thread. When the call responds back the virtual thread is again associated to any available platform thread to continue the processing. This unmounting off platform threads when not needed and mounting onto them when needed is what makes virtual threads very efficient
In this article we will explore two Java features of asynchronous programming — CompletableFuture and Structured Concurrency and how they leverage virtual threads
Completable Future with Virtual Threads
CompletableFuture was introduced in Java 8 for asynchronous processing. Typically CompletableFuture creates platform threads for async processing. Can CompletableFuture use virtual threads is what we will see. The following example uses CompletableFuture to call a service multiple times in a loop in asynchronous fashion.
The LearningService is a simple service that just returns the current thread information before and after a certain delay. The LearningResource uses Executors to create platform threads or virtual threads dependending on the properties config. The CompletableFuture uses the executor created threads to call the LearningService method asynchronously 1000 times. After the asynchronous calls are complete, the values are obtained from the CompletableFutures.
@Service
public class LearningService {
public String getThreadInfo() throws InterruptedException {
String threadInfo = "Entry Thread : " + Thread.currentThread().toString();
Thread.sleep(2000);
threadInfo = threadInfo + ", Exit Thread : " + Thread.currentThread().toString();
return threadInfo;
}
}
@RestController
public class LearningResource {
@Autowired
private LearningService learningService;
@Value("${spring.threads.virtual.enabled:false}")
private boolean virtualThreadsEnabled;
@GetMapping(value = "learning/virtualthreads/completablefuture")
public ResponseEntity<HttpStatus> createThreadsUsingCF(){
System.out.println("Async processing using completable future !!!");
long startDateTime = System.currentTimeMillis();
ExecutorService executorService;
if (virtualThreadsEnabled)
executorService = Executors.newVirtualThreadPerTaskExecutor();
else
executorService = Executors.newFixedThreadPool(100);
List<CompletableFuture<String>> cfList = new ArrayList<>();
IntStream.range(0, 1000).forEach(i ->
cfList.add(CompletableFuture.supplyAsync(() -> {
try {
return learningService.getThreadInfo();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}, executorService))
);
cfList.stream().forEach(cf -> {
try {
System.out.println(cf.get());
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
});
long endDateTime = System.currentTimeMillis();
System.out.println("Time taken in milliseconds : " + (endDateTime-startDateTime));
return new ResponseEntity<>(HttpStatus.OK);
}
}When virtualThreadsEnabled is False, it uses the Executor service fixed thread pool of 100 threads to perform the asynchronous processing. It runs for 20 secs to call the service 1000 times using 100 threads from the threadpool and around 50 MB of memory
With virtualThreadsEnabled = True, it runs for 2 secs to call the service 1000 times using 37 threads and around 21 MB of memory
The above comparison shows that CompletableFuture is able to use virtual threads and handle asynchronous processing in a very efficient manner.
Also the below logs of the entry and exit threads info provided by the LearningService shows the mounting/unmounting of a virtual threads. For example : virtual thread #10080 is mounted on a platform thread ForkJoinPool-1-worker-9 before the delay. After the delay it could be seen that it has unmounted and mounted on a different platform thread ForkJoinPool-1-worker-7. This way a virtual thread allows the platform thread to be freed up for other processes when the virtual thread is waiting. Hence using virtual threads is very suitable for processes that have low CPU usage and high I/O or network calls.
Entry Thread : VirtualThread[#10080]/runnable@ForkJoinPool-1-worker-9, Exit Thread : VirtualThread[#10080]/runnable@ForkJoinPool-1-worker-7
Entry Thread : VirtualThread[#10081]/runnable@ForkJoinPool-1-worker-8, Exit Thread : VirtualThread[#10081]/runnable@ForkJoinPool-1-worker-6
Entry Thread : VirtualThread[#10082]/runnable@ForkJoinPool-1-worker-10, Exit Thread : VirtualThread[#10082]/runnable@ForkJoinPool-1-worker-9
Entry Thread : VirtualThread[#10083]/runnable@ForkJoinPool-1-worker-2, Exit Thread : VirtualThread[#10083]/runnable@ForkJoinPool-1-worker-13
Entry Thread : VirtualThread[#10084]/runnable@ForkJoinPool-1-worker-12, Exit Thread : VirtualThread[#10084]/runnable@ForkJoinPool-1-worker-7
Entry Thread : VirtualThread[#10085]/runnable@ForkJoinPool-1-worker-3, Exit Thread : VirtualThread[#10085]/runnable@ForkJoinPool-1-worker-9CompletableFuture with Virtual Threads — Alternate Approach
An alternative approach to use CompletableFuture with virtual threads is to use the Spring boot 3.2.x provided AsyncTaskExecutor class. This class wraps both CompletableFuture and also the Executor for virtual threads thus providing a single bean. The following example shows a configuration to create a AsyncTaskExecutor bean. In the LearningResource this bean is used to create CompletableFuture that automatically uses virtual threads.
@EnableAsync
@Configuration
@ConditionalOnProperty(
value = "spring.thread-executor",
havingValue = "virtual"
)
public class VirtualThreadConfig {
@Bean(TaskExecutionAutoConfiguration.APPLICATION_TASK_EXECUTOR_BEAN_NAME)
public AsyncTaskExecutor asyncTaskExecutor() {
return new TaskExecutorAdapter(Executors.newVirtualThreadPerTaskExecutor());
}
@Bean
public TomcatProtocolHandlerCustomizer<?> protocolHandlerVirtualThreadExecutorCustomizer() {
return protocolHandler -> {
protocolHandler.setExecutor(Executors.newVirtualThreadPerTaskExecutor());
};
}
}@Service
public class LearningService {
public String getThreadInfo() throws InterruptedException {
String threadInfo = "Entry Thread : " + Thread.currentThread().toString();
Thread.sleep(2000);
threadInfo = threadInfo + ", Exit Thread : " + Thread.currentThread().toString();
return threadInfo;
}
}
@RestController
public class LearningResource {
@Autowired
private LearningService learningService;
@Autowired
@Qualifier("applicationTaskExecutor")
private AsyncTaskExecutor asyncTaskExecutor;
@GetMapping(value = "learning/virtualthreads/asynctaskexecutor")
public ResponseEntity<HttpStatus> createThreadsUsingAsyncExec(){
System.out.println("Async task executor using virtual threads !!!");
long startDateTime = System.currentTimeMillis();
List<CompletableFuture<String>> cfList = new ArrayList<>();
IntStream.range(0, 1000).forEach(i ->
cfList.add(asyncTaskExecutor.submitCompletable(() -> learningService.getThreadInfo()))
);
cfList.stream().forEach(cf -> {
try {
System.out.println(cf.get());
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}
);
long endDateTime = System.currentTimeMillis();
System.out.println("Time taken in milliseconds : " + (endDateTime-startDateTime));
return new ResponseEntity<>(HttpStatus.OK);
}Structured Concurrency
Structured Concurrency is a preview feature in Java 21. This enables to have a parent child relationship in asynchronous threads. StructuredTaskScope is the class that supports splitting a task into several concurrent sub tasks, each sub task executing using a virtual thread, handling the subtasks success or errors. This ensures that all subtasks must complete before the main task continues.
StructuredTaskScope is a better option compared to CompletableFutures and it inherently uses virtual threads. The following example using StructuredTaskScope produces the same desired result as a CompletableFuture with virtual threads and with better control over sub tasks completion
@GetMapping(value = "learning/virtualthreads/structuredconcurrency")
public ResponseEntity<HttpStatus> performStructuredConcurrencyUsingVirtualThreads() throws InterruptedException {
System.out.println("Structured concurrency using virtual threads !!!");
long startDateTime = System.currentTimeMillis();
var scope = new StructuredTaskScope.ShutdownOnFailure();
List<Supplier<String>> supplierList = new ArrayList<>();
IntStream.range(0, 1000).forEach(i ->
supplierList.add(scope.fork(() -> learningService.getThreadInfo()))
);
scope.join().throwIfFailed(ScopeException::new);
supplierList.stream().forEach(supplier -> {
System.out.println(supplier.get());
}
);
long endDateTime = System.currentTimeMillis();
System.out.println("Time taken in milliseconds : " + (endDateTime-startDateTime));
return new ResponseEntity<>(HttpStatus.OK);
}
Using StructuredConcurrency, it runs for 2 secs to call the service 1000 times using 37 threads and around 24 MB of memory
Conclusion
CompletableFutures using virtual threads give much better performance than using fixed thread pool
Structured Concurrency provides the dual benefit of using virtual threads for better performance and also having better control over asynchronous sub tasks
