Daily Archives: March 18, 2025

[javaone 2025] java for ai

Posted on by
Reply

Speakers: Paul Sandoz

See the table of contents for more posts

General

  • Not going to add XML to the platform as it changes. Scala did that and thinks wrong choice
  • Goals: developer productivity and program performance
  • Many features meet demands of AI but useful for other things
  • “All web companies grow up to be Java companies”. Want to make all AI companies grow up to be Java companies as well
  • AI on Java is harder than it should be

Panama – Foreign Function and Memory (FFM) API

  • Optimal usage for foreign (off heap) memory
  • Better interoperation with foreign (native) APIS
  • Let’s map to native libraries
  • ex: used for Matrix API using native BLIS. The matrix API is Java (for linear algebra)
  • jextract gives a C like interface in Java. Still have do deal with allocating memory
  • Zero copy memory since all memory is held natively on the heap. JVM is not involved

Panama – Vector API

  • SIMD (single instructions, multiple data) programming for optimal CPU utilization
  • Better number crunching
  • Useful for high performance data parallel algorithms
  • Incubating a long time because need value classes to be released
  • Example: https://github.com/mukel/llama3.java
  • Testcase to improve the runtime compiler.
  • Not as fast as native algorithm but might be good enough one day.
  • Show performance improvement from using Vector AI with a demo. The “before” was so slow it was hard to watch. The “after” was essentially at the speed of reading.

Valhalla – Value classes and objects

  • Optimal utilization of on heap member
  • Enables more kinds of numbers
  • Runtime will optimize
  • More kinds of numbers — ex:Float16, Decimal64, Complex, Interval, etc
  • Incubating

Babylon – Code Reflection

  • Interoperates with foreign programming models – ex onnx
  • Better use of hardware and better number crunching
  • For foreign programming models aka not the one for the Java Language Specifications. Ex: GPU, autoparallelization of for loops
  • New Java code model — symbolic in memory representation of java code using a tree like structure. Like the compiler’s AST but more suited for analysis and transformation

My take

I like that he was able to go into more detail than we got into at the keynote. Good demo. Some of it was too advanced for me (not an exert on the matrix stuff), but I learned a lot. I missed a bunch. I went to google something I didn’t know and fell down a rabbit hole. He also showed Onnx which we saw during the keynote.

[javaone 2025] next level features of langchain4j

Posted on by
Reply

Speakers: Lize Raes, Mohamed Abderrahman

See the table of contents for more posts

Concepts

  • SystemMessage – instructions
  • ContentRetriever – context
  • Tools – function calling
  • UserMessage – user to LLM
  • AiMessage – LLM to user
  • ChatMemory

5 Levels towards AGI

  • Can perform work of entire orgs of people
  • Can create new innovations
  • Can take actions on users behalf
  • Can solve basic problems like a PHD with tools
  • Current AI like ChatGPT that takes with humans

Options

  • LLM manages step transitions in state machine – can jump states when unexpected requests, flexibility, but risky
  • Code manages step transitions – any complexity possible, reliable, separation of concerns, tailored model size. However, not flexible. can’t deal with unexpected scenarios and more work to write.

RAG

  • Retrieval Augmented Generation
  • Fetch info relevant to request and send information to LLM
  • Advanced RAG features
  • Retrieval Augmentor in addition to retriever
  • LLM writes query
  • Adds info/context
  • Need to measure performance of model. Compare across models
  • MCP (Model Context Protocol)

Steps in code:

  • Create document content retriever – can limit scope. Ex: scientific literature
  • Create web search content retriever
  • Create SQL database content retriever

Guardrails and Moderation

  • Guardrails add limits. Ex: list examples of queries that shouldn’t be allowed
  • Moderation – checks if message violent, etc. Can use a different model for validations
  • LLMs are more sensitive to examples than instructions

Testing approaches

  • Test human evaluation (thumbs up/down)
  • AI assisted

Websites

  • swebench.com – closes github issues
  • llm-price.com – shows prices per token and per million tokens
  • JUnit Pioneer – test retry
  • Examples from session: https://github.com/LizeRaes/ai-drug-discovery

My take

Excellent examples. The real world scenario of diseases/antigens/antibodies was good. Good concepts and great demo. Showing Prometheus/Graphana was good as well.

[javaone 2025] opening keynote

Posted on by
Reply

See the table of contents for more posts

Chad Arimura

Started with history including pictures from the first JavaOne in 1996 and what Duke looked like then. Fast forward to present day. Java 24 launched this morning

Scott McNealty (retired but was here for the beginning) and Colt McNealy (Scott’s son)

Scott’s reminisced about the start of Java. Were originally planning on making a clicker. Never did but did better. Talked about licensing stuff to companies for a fee. Virtualization; creating a mess.

In the background was Fortune magazine’s JavaMan cartoon of Scott McNealy needing to stop Bill Gates from dominating the corporate universe.

Colt took AP computer science, went to Stamford and then got a real job. Real word harder than college where everything runs on one laptop. Created open source Lighthouse to get insights into threads, etc

Georges Saab

  • 94 of Fortune 100 run Java
  • Small companies use Java too
  • ex: Uber, Netflix, LinkedIn
  • 6 billion java cars deployed each year – ex: smartphones have two Java parts, credit cards, RFID cards, car
  • 15 releases under 6 month cadeence (since Java 10)

Mark Cavage

  • “crazy Mark” to disambiguate which Mark (vs Mark Reinhold). “Crazy” because proposed six month cadence
  • Now at Docker
  • 8M Java developers on Docker
  • Can run LLMs natively in Docker: docker model run ai/llama3.2:1b

Georges Saab

  • Not every change gets a JEP
  • Read release notes
  • Performance

Mingmin Chen (Head of Uber’s real time search platform)

  • Asked how many people used Uber in last 30 days. It was almost everyone.
  • Use Kafka, Flink, Pinot, judi, spark, presto, open search
  • Generational ZGC to reduce memory usage. Also reduced CPU

Georges Saab

AI

  • code generation/assistance
  • model integration for business apps (langchain4j)
  • model training (panama, valhalla, babylon)

New learners

  • simple source files
  • instance main methods
  • single/multi file source code launcher
  • script ready shebang upport

Missed name but someone from College Board and also Heather Stevens

  • AP Computer Science A being revised in Fall 2025 from Java 8
  • Encouraging teachers each year to use latest version of java
  • Exam has used Java for 0ver 20 years [I took it when Pascal and C++)
  • A third of schools with one AP class have Computer Science A

Mark Reinhold

  • Developer vs steward
  • Reading is more important than writing
  • Simplicity matters
  • Java should be one language with the same meaning everywhere
  • Java has never been about write once and throwaway code. [that’s what vibe coding is I think?]
  • “first do no harm”
  • Compatibility – source, binary, behavioral, migration, intellectual (build on existing knowledge)
  • Project Amber “paving the on ramp”.

JEPS 494 and 495 with more to come

  • Showed evolving the classic Hello World to use an instance main, removing the args, removing class declaration, import so can write println(). Final form is 
void main() {
   println("Hello World!"); 
}
  • import module java.base; to import everything in module in one shot
  • chmod +x fileName; ./fileName to run

Garbage Collection

  • Compared garbage collectors. ZGC from Java 15 is extremely low latency..
  • In Java 17, got to sub millisecond pause times.
  • ZGC doesn’t scale well . Fixed in Java 21 by making generational
  • Still need multiple garbage collectors because GC uses a lot of memory.
  • Java 24 – improved latency in Parallel an G1 garbage collectors

Paul Bakker (Netflix)

  • Use GraphQL, Spring Boot, Kafka, gRPC, cassandra, etc
  • Had client timeouts from pause times. Switched to ZGC and saw no pause times so clients have less errors

Mark Reinhold

Project Leyden

  • improve startup time
  • Move work earlier or later
  • Goal is to avoid new contrainst on existing code, without changing spec and without making Java less dynamic
  • Instead use training runs
  • Training: java -XX:AOTMode=record -XX:AOTConfiguration-app.aotconf…
  • Assembly: java -XX:AOTMode=create -XX:AOTConfiguration-app.aotconf.-XX:AOTCache=-app.aot ….
  • Execution: java -XX:AOTCache=-app.aot ….
  • Can do a lot of work up front,. Too long to list. Ex: link classe (in Java 24) and more in future versions of Java
  • Took about 30% of the original time before Leyden

Native libraries

  • hard to use JNI by design
  • Useful for crypto, ML, etc
  • Project Panama add Foreign Function and Memory API. Use jextract to automatically generate bindings rom the C/C++ header file.

Paul Sandoz

  • Function is code and weights
  • Training computes the weights
  • ONNX – open neural network exchange for ML. Includes interoperable format for ML models and runtime for executing them
  • Use Panama and jextract to bind to onnx.h file

Mark Reinhold

More pain points

  • Quantum computing will break RSA. Preparing with Key encapsulation, key derivation, quantum resistant mechanism/algorithms. Will eventually back port to past LTS releases.
  • The stream API isn’t fully extensible. Java 24 stream gatherers
  • My favorite bytecode library breaks on Java 21 – “stop using it” . Class file API
  • Writing raw HTML in JavaDoc is tedious. Now can use markdown.
  • You aren’t delivering new features quickly enough – integrity by default. Need to be able to refactor JDK internals. Deprecate/remove things not used

Bruno Borges and Mark Heckler (Microsoft)

  • VS Code, (3 million monthly downloads) GitHub, Azure (95% of Fortune 500 customers)
  • AI agent – create a plan, retrieve context, perform an action
  • Nice demo of Copilot chat in VS Code. Also showed Codespaces
  • Microsoft has 2.5 million JMVs running internally. Includes Minecraft, Bing, etc
  • Minecraft uses Java 21

Chad Arimura

  • dev.java – tutorials for latest features. Meant for professionals
  • learn.java – new. For teachers and students

My take

I like that they covered a lot and quickly. Some things were interesting like the AP changes but only needed a few minutes to cover so it was great that is what they got. While the language parts weren’t new to me, it was well covered. Also nice Mark threw in a few cat pictures. I hadn’t heard of Project Leyden or GRPC along with some of the new features. And I wasn’t familiar with the native binding changes. Nice to learn a bunch of new stuff. It was nice to see a few demos as well