TornadoVM, Llama, GraalVM, Valhalla, Jakarta EE: Bingo Edition - JVM Weekly vol. 134
Today, in JVM Weekly, we are playing bingo with the most interesting JVM Technologies. You will have updates about TornadoVM, GraalVM, but also Valhalla... and interesting updates from Jakarta EE!
We return today to my favorite type of issue - one that “requires” nothing from me, neither because of a fixed editorial format (“Rest of the Story” with Release Radar, guest issues of Foojay.io), nor tied to any event happening at a specific moment (JDK 25 Rampdown Phase).
Don’t get me wrong - those are fun to create, they have their place, and they give this little project a nice rhythm. But there’s always something refreshing about moments when nothing really directs or constrains me, so I can pick topics simply because I find them interesting and want to share them with you. That’s why today’s issue is exactly that: I might call it “Classic.”
Four topics, each different, all super-interesting from my perspective… and probably quite niche. But as they say in Hollywood: one for me, one for them.
So today we start with TornadoVM!
1. TornadoVM brings Llama support
I was trying to recollect when I first wrote about TornadoVM, and I think it was back in August 2023. However, I truly got interested in the project when I heard about it on Adam Bien’s podcast with Juan Fumero. Back then, “Java and GPU” in the same sentence sounded like a stretch, and the whole project felt more like a cool technical flex than a practical tool. But look how far we’ve come: TornadoVM - once an academic experiment from Manchester - just released version 1.1, and the brand-new GPULlama3.java repo shows it can run 1B, 3B, 7B, and 8B parameter language models, spitting out over a hundred tokens per second - all in pure Java, with zero user-level JNI and zero CUDA lines.
Since then, Juan has been a guest on the Bien’s podcast more than once, last time middle of May.
The story began a decade ago at BeeHive Lab, University of Manchester. The grad students there were tweaking Graal’s JIT, discovered they could emit not only x86 code but also OpenCL C, and so the idea of a “heterogeneous VM” was born—hence Tornado. The early prototype worked, but it was more of an adventurous explorer’s toy than a product of high quality.
Between 2020 and 2023 the project went open-source, added PTX and SPIR-V back-ends to target Nvidia, OpenCL, and Intel GPUs, shipped its first plugin-based profiler, and even an IntelliJ IDEA plugin that checks in real time whether your method can map to a GPU. Each release came with more examples: from classic multiply-add kernels to tiny MNIST-style neural networks. Yet limitations lurked everywhere: memory copies ate most of the compute gains.
January’s TornadoVM 1.0 closed the purely academic chapter by having its first production deployment. TornadoVM is being used by the GAIA mission of the European Space Agency to accelerate the processing of the vast amount of data collected from the spaceship while doing a 3D mapping of our Milky Way galaxy. In addition, TornadoVM 1.0 brought production-level quality additions such as: JDK 21 support, Docker images, official docs - and under the hood, shared buffers for execution plans and fixes for memory leaks. But mixed-precision and persistent kernel caches were still missing.
Of course, technical goodies alone won’t persuade skeptics - you need a killer demo. And here comes GPULlama3.java on GitHub: full Llama 3 (and soon Mistral) inference in pure Java, no native libs, with TornadoVM dynamically converting methods in the math package into PTX and OpenCL kernels on the fly. The first numbers: around 100 tokens/sec on an RTX 5090 in FP16 - 2× to 6× faster (on Mistral) than the same app on CPU Vector API. Even better, the same JAR runs on Apple Silicon and Intel GPUs - just switch back-ends with a command-line flag.
For cloud microservices folks, this might seem a curiosity, but it unlocks a few exciting possibilities. First, no more JNI bridges to CUDA libraries: frameworks like DJL can treat TornadoVM as just another runtime, keeping you safely in the JDK world. Second, Panama’s MemorySegments finally make sense on the GPU: pin them to device buffers to eliminate copy overhead. The third, perhaps least obvious but most wallet-friendly: TornadoVM team is working on a separate sub-project that will enable the JEP’s 515 AOT of GPU kernels for its integration with GraalVM.
TornadoVM is production-ready, with a stable API, old problems solved, and a show-stopping demo that grabs attention. Does this mean GPUs will become OpenJDK’s next default backend? That answer will take a few more JDK releases (likely thanks more to Project Babylon than TornadoVM). But if we ever dreamed that “write once, run anywhere” would include GPUs too, we’ve just taken a meaningful step.
Small step for a project, big step for the community.
PS: A fun fact about today’s thumbnail of this section: when I asked for an image (I admit I don’t draw them myself but like to imagine them - it’s a lot of fun) of a tornado made of llamas rampaging through Manchester’s streets, 4o for some reason added rain. Team TornadoVM - does it always rain in Manchester? 😄
2. Project Crema “opens” GraalVM
Now let’s swing to the other end of the “Write Once, Run Anywhere” spectrum - GraalVM. Here, too, we’re breaking key limitations.
Since GraalVM Native Image debuted in 2019 promising Java native binaries with millisecond cold-start times - and indeed delivering them - its Achilles' heel has been JVM magic itself: dynamic class loading, runtime-generated proxies, hot-loaded plugins. Native Image lives in a closed world: everything must be known at build time. Anyone who's tried running Hibernate without a dozen reflection.json configs knows the pain.
The GraalVM team has been working on further improving the developer experience: adding a JDWP debugger so you don't have to wrestle GDB, Native Image Layers for composable AOT slices that share memory and speed up builds. But the closed-world assumption remained. If your app wanted to load a ByteBuddy proxy at runtime or pull a plugin from disk… you were out of luck. We built workarounds, wrote config generators, and accepted that fast startup cost us flexibility.
And now, like a doppio shot of espresso cutting through this intro, comes Project Crema. Announced in early June in GitHub PR #11327 its goal is simple: puncture the closed-world model so a native JVM app can load and execute new classes at runtime.
How? Crema layers a mini bytecode interpreter atop the existing AOT code, handling anything not baked into the image. These dynamically loaded classes can still call AOT-compiled methods, and in future iterations they'll get an optional JIT too.
In practice, this means code-generation libs (Lombok, MapStruct, Spring AOT test-slices), plugin engines, even javac inside an IDE no longer need to be known at image-build time. Crema won't be on by default - you'll still get lean, hermetic binaries - but when you need flexibility, flip a build flag and opt into "open world."
If you hear echoes of Oracle's attempts to marry static images with a live JVM - you're spot on. Crema builds on GraalVM's existing layers, debuggers, and tooling to make native Java friendlier, taking a step many have awaited for five years. It opens new Native Image use cases: plugin servers, containerized IDEs, you name it.
There's a catch: Crema won't retrofit dynamic loading into images already built, nor let you switch on-the-fly between interpreted and compiled modes for classes included at AOT time. But even with those limits, it's an enticing vision. Imagine a Spring Boot CLI built as a tiny exec yet still able to load starters without rebuilding, or a plugin platform where users drop Java addons into a folder while the core app still boots in 40 ms.
For Native Image "configuration suffering" sceptics, Crema might be a turning point. If the interpreter lands and a lightweight JIT follows after, we'll enter a hybrid half-AOT, half-JVM world - still lightning-fast at startup compared to classic HotSpot. For now, brew another coffee, watch the Graal repo, and see how Java's new “Crema” flavor unfolds.
3. Valhalla and a new “Parametric VM” proposal
When Brian Goetz launched Valhalla a decade ago (I will spare you the old joke about how long we’ve been waiting), he promised two big things: primitive classes - identity-less objects packing into registers like numbers - and a “parametric JVM” to resurrect generic type information at runtime. So far, all energy has funneled into the first promise: inline/value classes have JEPs and preview builds, while generics still live under erasure. How can the VM work magic on types that vanish at compile time?
That’s why Rémi Forax’s email from June 2nd to valhalla-spec-experts is so intriguing. Forax - never shy about contrarian ideas - asked: “What’s the absolute minimum to let standard collections specialize at runtime?” In other words: how can ArrayList<LocalDate> behave differently in memory layout than ArrayList<String>, with that difference visible deep down?
Instead of new language syntax, he proposes extending the class-file format and two JVM instructions. First, a new method modifier ACC_SIDE_VALUE marks methods that accept an extra “side value” object carrying the actual type parameters. It hooks into the existing wide bytecode prefix: to the VM it’s just a prefix on invokestatic. Java compilation stays erasure-only, but the side channel brings type info into the call. Inside the method, you read it via a new currentSideValue() API; if you invoke without wide, you get null, so old code is untouched.
Second, a class flag ACC_PARAMETRIC plus a new anew_parametric opcode mirror that trick for constructors: at object creation you inject metadata about memory layout or generic specialization, readable via currentMetadata(). Thus, even erasure-compiled bytecode can tell the VM to pack fields tightly for specific type arguments.
The beauty of Forax’s pitch is that the Java compiler needn’t know any of this. Only libraries wanting specialization (say, a future java.util that’s “specialization aware”) mark their methods and classes. Everything else just works. And as Forax notes, the JIT can propagate side values through the call graph as constants, enabling aggressive inlining without if-ladder overhead.
Will this solve reification? Probably not overnight - the proposal is just a few pages and doesn’t tackle code-sharing or classloader limits. But it strikes at the heart of Valhalla’s stalled “Parametric VM”: rather than hauling full generic info from source to runtime, preserve erasure and give the VM a little side pocket for quick type data. For a project long associated with brick-by-brick JVM foundational work, this “side value” feels refreshingly pragmatic.
If you’ve missed the past half-decade of vanishing “Parametric VM” drafts, Forax’s email sounds like a handbrake turn towards incremental evolution: add a couple of flag bits, see what the JIT can do when it has side values via wide. And if libraries find real gains - say, a date list that sits in memory like a long array - that channel can widen.
Will it land? I’m skeptical, but the idea is intriguing enough to share. After all, the email ends simply: “I just want to get the conversation started.”
4. Jakarta EE 12 is poised to breathe new life into the project.
Finally, a Jakarta EE team announcement.
From day one Jakarta EE felt like a marathon on a construction site: first move Java EE’s entire codebase into Eclipse Foundation, then painstakingly rewrite every javax.* to jakarta.*, rebuild the tool and app-server ecosystem. Only then could you notice the engine running underneath was from a bygone era - Java EE 8 - and its tech debt grew heavier with each JDK release.
That stale foundation relegated the platform to the fringes of modern Java. So over the past two years hundreds of volunteers split the TCK tests into smaller modules, replaced obsolete APIs, and patched holes dating back before containers. Version 11 -now in GA testing - was the payoff: most tech debt repaid, a solid base for bigger ideas. With the old furniture gone, you can dream again.
Hence, the Jared Anderson and Edward Burns bold claim published in Eclipse newsletter’s: Jakarta EE 12 won’t walk, it will run. First up: a JDK version jump - every component must support at least Java 21 (and optionally 25), nudging library authors to use records, virtual threads, and all the new language goodies. This also nudges users off soon-unsupported Java 8 and 11, so enterprise projects can’t cling to legacy forever.
Next, configuration: Spring has let you swap properties without recompilation for over a decade, but Jakarta lacked an equal. In 12 they’re unifying external param injection for simpler, cloud-native environments and secret management.
Simultaneously, they’re harmonizing the query layer: today Jakarta Data’s JPQL fork and Jakarta NoSQL’s custom dialect diverge, so the plan is to carve shared bits into a common spec - bringing relational and document persistence closer. A similar informal proposal for Jakarta HTTP aims for one unified HTTP request/response API, instead of juggling Servlet and JAX-RS classes.
Modernization also means farewells: the old Application Client, relic of fat-client days, will be deprecated and eventually removed. They’ll also clean out the last traces of SecurityManager, which newer JDKs have disabled by default and which only muddle the API.
Taken together, these moves lightened the platform’s baggage so it can sprint. Version 11 finished the tech-debt workout; version 12 is the first true step forward, in step with mainstream Java. For firms stuck on WebLogic and Java 8, it’s a clear signal: time to move. And for the open-source community, an invitation to join the discussion now, before the dusty ideas get set in stone.
We’ll see - maybe Jakarta EE 12 really will breathe new life into the platform.
