Speakers: Paul Sandoz

See the table of contents for more posts

---

General

• Not going to add XML to the platform as it changes. Scala did that and thinks wrong choice
• Goals: developer productivity and program performance
• Many features meet demands of AI but useful for other things
• “All web companies grow up to be Java companies”. Want to make all AI companies grow up to be Java companies as well
• AI on Java is harder than it should be

Panama – Foreign Function and Memory (FFM) API

• Optimal usage for foreign (off heap) memory
• Better interoperation with foreign (native) APIS
• Let’s map to native libraries
• ex: used for Matrix API using native BLIS. The matrix API is Java (for linear algebra)
• jextract gives a C like interface in Java. Still have do deal with allocating memory
• Zero copy memory since all memory is held natively on the heap. JVM is not involved

Panama – Vector API

• SIMD (single instructions, multiple data) programming for optimal CPU utilization
• Better number crunching
• Useful for high performance data parallel algorithms
• Incubating a long time because need value classes to be released
• Example: https://github.com/mukel/llama3.java
• Testcase to improve the runtime compiler.
• Not as fast as native algorithm but might be good enough one day.
• Show performance improvement from using Vector AI with a demo. The “before” was so slow it was hard to watch. The “after” was essentially at the speed of reading.

Valhalla – Value classes and objects

• Optimal utilization of on heap member
• Enables more kinds of numbers
• Runtime will optimize
• More kinds of numbers — ex:Float16, Decimal64, Complex, Interval, etc
• Incubating

Babylon – Code Reflection

• Interoperates with foreign programming models – ex onnx
• Better use of hardware and better number crunching
• For foreign programming models aka not the one for the Java Language Specifications. Ex: GPU, autoparallelization of for loops
• New Java code model — symbolic in memory representation of java code using a tree like structure. Like the compiler’s AST but more suited for analysis and transformation

My take

I like that he was able to go into more detail than we got into at the keynote. Good demo. Some of it was too advanced for me (not an exert on the matrix stuff), but I learned a lot. I missed a bunch. I went to google something I didn’t know and fell down a rabbit hole. He also showed Onnx which we saw during the keynote.