Tag Archives: ai

[javaone 2025] java for ai

Posted on by
Reply

Speakers: Paul Sandoz

See the table of contents for more posts

General

  • Not going to add XML to the platform as it changes. Scala did that and thinks wrong choice
  • Goals: developer productivity and program performance
  • Many features meet demands of AI but useful for other things
  • “All web companies grow up to be Java companies”. Want to make all AI companies grow up to be Java companies as well
  • AI on Java is harder than it should be

Panama – Foreign Function and Memory (FFM) API

  • Optimal usage for foreign (off heap) memory
  • Better interoperation with foreign (native) APIS
  • Let’s map to native libraries
  • ex: used for Matrix API using native BLIS. The matrix API is Java (for linear algebra)
  • jextract gives a C like interface in Java. Still have do deal with allocating memory
  • Zero copy memory since all memory is held natively on the heap. JVM is not involved

Panama – Vector API

  • SIMD (single instructions, multiple data) programming for optimal CPU utilization
  • Better number crunching
  • Useful for high performance data parallel algorithms
  • Incubating a long time because need value classes to be released
  • Example: https://github.com/mukel/llama3.java
  • Testcase to improve the runtime compiler.
  • Not as fast as native algorithm but might be good enough one day.
  • Show performance improvement from using Vector AI with a demo. The “before” was so slow it was hard to watch. The “after” was essentially at the speed of reading.

Valhalla – Value classes and objects

  • Optimal utilization of on heap member
  • Enables more kinds of numbers
  • Runtime will optimize
  • More kinds of numbers — ex:Float16, Decimal64, Complex, Interval, etc
  • Incubating

Babylon – Code Reflection

  • Interoperates with foreign programming models – ex onnx
  • Better use of hardware and better number crunching
  • For foreign programming models aka not the one for the Java Language Specifications. Ex: GPU, autoparallelization of for loops
  • New Java code model — symbolic in memory representation of java code using a tree like structure. Like the compiler’s AST but more suited for analysis and transformation

My take

I like that he was able to go into more detail than we got into at the keynote. Good demo. Some of it was too advanced for me (not an exert on the matrix stuff), but I learned a lot. I missed a bunch. I went to google something I didn’t know and fell down a rabbit hole. He also showed Onnx which we saw during the keynote.

[javaone 2025] next level features of langchain4j

Posted on by
Reply

Speakers: Lize Raes, Mohamed Abderrahman

See the table of contents for more posts

Concepts

  • SystemMessage – instructions
  • ContentRetriever – context
  • Tools – function calling
  • UserMessage – user to LLM
  • AiMessage – LLM to user
  • ChatMemory

5 Levels towards AGI

  • Can perform work of entire orgs of people
  • Can create new innovations
  • Can take actions on users behalf
  • Can solve basic problems like a PHD with tools
  • Current AI like ChatGPT that takes with humans

Options

  • LLM manages step transitions in state machine – can jump states when unexpected requests, flexibility, but risky
  • Code manages step transitions – any complexity possible, reliable, separation of concerns, tailored model size. However, not flexible. can’t deal with unexpected scenarios and more work to write.

RAG

  • Retrieval Augmented Generation
  • Fetch info relevant to request and send information to LLM
  • Advanced RAG features
  • Retrieval Augmentor in addition to retriever
  • LLM writes query
  • Adds info/context
  • Need to measure performance of model. Compare across models
  • MCP (Model Context Protocol)

Steps in code:

  • Create document content retriever – can limit scope. Ex: scientific literature
  • Create web search content retriever
  • Create SQL database content retriever

Guardrails and Moderation

  • Guardrails add limits. Ex: list examples of queries that shouldn’t be allowed
  • Moderation – checks if message violent, etc. Can use a different model for validations
  • LLMs are more sensitive to examples than instructions

Testing approaches

  • Test human evaluation (thumbs up/down)
  • AI assisted

Websites

  • swebench.com – closes github issues
  • llm-price.com – shows prices per token and per million tokens
  • JUnit Pioneer – test retry
  • Examples from session: https://github.com/LizeRaes/ai-drug-discovery

My take

Excellent examples. The real world scenario of diseases/antigens/antibodies was good. Good concepts and great demo. Showing Prometheus/Graphana was good as well.

Free Copilot.. Which IDEs shall I use

Posted on by
Reply

Earlier this month, Microsoft announced free GitLab Copilot. You get 2000 autocompletes a month and 50 chat messages. The idea is clearly that you’ll start relying on it and then pay.

I installed and configured in both VS Code and IntelliJ. Setup was easy. I had to authorize the IDE by entering a code on github.com that the IDE gave me. It also gave me the link to enter it so pretty easy.

I also installed it on PyCharm and quickly uninstalled it. Copilot was being too helpful and I need to practice more typing the idioms as I get better at Python. (This is my fifth time learning Python. Every time I get decent at it, I don’t use it for a long time and forget. Maybe this time it’ll take!)

I do use AI periodically to ask how to improve the Python code I write (at home on personal stuff). But I can do that outside my IDE.

I like that Microsoft is doing this. They’ve had a free trial of CoPilot for a long time, but that is time limited. I like that this one is usage limited. So if you don’t use it much, you have it when you need it and still free.