Good timing. I was actually reading the official docs the other day on virtual threads, and it has a big section decidicated to rate limiting virtual threads:

"The hardest thing to internalize about virtual threads is that, while they have the same behavior as platform threads they should not represent the same program concept."

...

"Sometimes there is a need to limit the concurrency of a certain operation. For example, some external service may not be able to handle more than ten concurrent requests. Because platform threads are a precious resource that is usually managed in a pool, thread pools have become so ubiquitious that they're used for this purpose of restricting concurrency, "

...

"But restricting concurrency is only a side-effect of thread pools' operation. Pools are designed to share scarce resources, and virtual threads aren’t scarce and therefore should never be pooled!

"When using virtual threads, if you want to limit the concurrency of accessing some service, you should use a construct designed specifically for that purpose: the Semaphore class."

...

"Simply blocking some virtual threads with a semaphore may appear to be substantially different from submitting tasks to a fixed thread pool, but it isn't. Submitting tasks to a thread pool queues them up for later execution, but the semaphore internally (or any other blocking synchronization construct for that matter) creates a queue of threads that are blocked on it that mirrors the queue of tasks waiting for a pooled thread to execute them. Because virtual threads are tasks, the resulting structure is equivalent"

"Even though you can think of a pool of platform threads as workers processing tasks that they pull from a queue and of virtual threads as the tasks themselves, blocked until they may continue, the underlying representation in the computer is virtually identical. _Recognizing the equivalence between queued tasks and blocked threads will help you make the most of virtual threads._" (emphasis mine)

So it was cool seeing him arrive at the same conclusion (or he read the docss). Either way it was a nice timely article.

Link to the docs: https://docs.oracle.com/en/java/javase/21/core/virtual-threa...