Speakers: Lize Raes, Mohamed Abderrahman

See the table of contents for more posts

---

Concepts

• SystemMessage – instructions
• ContentRetriever – context
• Tools – function calling
• UserMessage – user to LLM
• AiMessage – LLM to user
• ChatMemory

5 Levels towards AGI

• Can perform work of entire orgs of people
• Can create new innovations
• Can take actions on users behalf
• Can solve basic problems like a PHD with tools
• Current AI like ChatGPT that takes with humans

Options

• LLM manages step transitions in state machine – can jump states when unexpected requests, flexibility, but risky
• Code manages step transitions – any complexity possible, reliable, separation of concerns, tailored model size. However, not flexible. can’t deal with unexpected scenarios and more work to write.

RAG

• Retrieval Augmented Generation
• Fetch info relevant to request and send information to LLM
• Advanced RAG features
• Retrieval Augmentor in addition to retriever
• LLM writes query
• Adds info/context
• Need to measure performance of model. Compare across models
• MCP (Model Context Protocol)

Steps in code:

• Create document content retriever – can limit scope. Ex: scientific literature
• Create web search content retriever
• Create SQL database content retriever

Guardrails and Moderation

• Guardrails add limits. Ex: list examples of queries that shouldn’t be allowed
• Moderation – checks if message violent, etc. Can use a different model for validations
• LLMs are more sensitive to examples than instructions

Testing approaches

• Test human evaluation (thumbs up/down)
• AI assisted

Websites

• swebench.com – closes github issues
• llm-price.com – shows prices per token and per million tokens
• JUnit Pioneer – test retry
• Examples from session: https://github.com/LizeRaes/ai-drug-discovery

My take

Excellent examples. The real world scenario of diseases/antigens/antibodies was good. Good concepts and great demo. Showing Prometheus/Graphana was good as well.