[2024 dev2next] virtual threads

Speaker: Piotr Przybyl @piotrprz and @piotrprz@mstdn.social

For more see the table of contents


Note: some things in this talk are in preview features. Might change/get better

Demo

  • Traditional demo of starting a thread and having it wait 10 seconds.
  • Ran 1K platform threads in just over 10 seconds. Ran 30K in 15 seconds. Ran 40K and it crashed due to “not enough space”.
  • Switched to virtual threads and no more resource constraints. Just over 10 seconds for 40K virtual threads. A million virtual threads took about 15 seconds
  • Awesome until see how fooled by demo
  • In the demo is sleeping, not using any resources
  • Redid demo with CPU work so no blocking.

Releases

  • released in Java 21 and previewed in 20-21
  • structured concurrency – preview in 23
  • Scoped values – preview in 22

Virtual threads

  • Great for I/O and waiting
  • No way to switch on and off globally in JDK
  • Don’t make threads faster; Make app scale better
  • Gave NYC example: drive to restaurant and park. Hardware resource (Car) not used. Vs taxi/Uber where don’t care where is when eating
  • If all cars were taxis, could move people faster. But one person doesn’t move faster by just using a taxi.
  • Said not his metaphor but likes it [I like it too]
  • Will not squeeze more juice from your CPU

Don’t Reuse

  • Reuse – cheap to start, create new
  • Pool – requires reusing

Don’t Pin

  • Like car “keep engine running”. Wastes resources, still have to pay.
  • I/O and synchronize pin in Java 21 when takes more than 20ms
  • Find with -Djdk.tracePinnedThreads=full and JFR events
  • toxiproxy – good for chaos engineering. Can set to half a second plus some jitter for variance (or anything you want)

Logging Double slit experiment – “how many lights are there”

  • Observing makes the answer different
  • Changed from 5 seconds of work to five one seconds and added logging.
  • Logging changing behavior because default handler is synchronous.
  • Less finished because opportunities for other threads to get in
  • It’s like everyone calling an Uber when a concert gets out.
  • Every time you do I/O, the thread blocks. ex: get out of taxi
  • Platform threads are different; they don’t block just because you do an I/O operation.

Other problems

  • Can’t ignore backpressure control
  • Ignore interrupts
  • merge with synchronized/native

Structured Concurrency (preview)

  • better “idioms” for multi threaded coded
  • helps eliminate thread leaks and cancellation delays
  • not replacing interruption with cancellation
  • showed waiting on two future. they don’t know the other gave up.
  • each future needs a reference to all other features so they know to give up when implemented manually
  • StructuredTaskScope.ShutdownOnFailure() { // two futures } // allows to know
  • scope.throwIfFailed() knows

Scoped Values (preview)

  • One way immutable ThreadLocals.
  • ThreadLocale is like a cache.
  • Bounded lifetime; visible in code
  • Results in simplified reasoning and improved performance
  • ScopedValue.where(x,y). run(() -> dostuff())
  • Can use Runnable or Callable
  • Can nest where() as long as you don’t mutate it

My take

This was interesting. I didn’t know pinned threads were a thing. Also good humor; well known debug pattern of “debug 1” 🙂