Category Archives: Conferences

[2024 dev2next] kafka

Posted on by
Reply

Speaker: Daniel Hinojosa (mastodon.social/@dhinojosa)

For more see the table of contents

Related tech

  • Piniot – OLAP
  • Input to kafka – kafka streams, akka streams, flink, spark streams
  • Connectors – ex: to database

Kafka

  • publish/subscribe queue
  • producer can also be a consumer

How looks inside

  • messages sharded
  • immutable data store
  • message gets an offset number in the partition.
  • data is temporary – specify retention size or time
  • don’t reuse offset numbers even after message delete
  • can only read/write to leader, not the replicas/followers

Message

  • Similar to a row/record
  • Just an array of bytes; format doesn’t matter
  • Message key is also an array of bytes. Only time something is guaranteed to be in order. Partitioner hashes key and maps to partition.

Batch

  • Group of messages
  • Every batch knows where each partition is going
  • Uses murmur2 for hashing
  • Can set batch size

Acknowledgements (Ack)

  • 0 – no ack; assume all is well; lowest latency
  • 1 – only goes to leader
  • all – all replicates must ack. Higher latency; safest. ex: bank transactions

Consumer

  • goal: scale to a large amount of different consumers without affecting performance
  • Consumers are not threadsafe
  • Consumer rebalance – mitigate when consumers go down
  • Settings: Isolation level

Producer

  • Settings: idempotent, transactions

Compaction

  • Retain message of same key where only latest message will be retained.
  • Cleaner thread does compaction
  • Can treat as events or tables
  • Tables treat Kafka as key/value database
  • LIkely don’t care about past with respect to table. Care about current/end state not everything that happened along the way.
  • Dirty – extra records

Stream processing

  • Everything is consumer/producer. Everything else is just a higher level
  • Stream groups
  • Java type stream methods – peek. foreach, groupByKey

My take

I used to know some of this, but had forgotten it so excellent review. And the new stuff was good too! I wish the screen had different resolution rather than relying completely on zooming in. Would have allowed to see some stuff while running, live code changing, and the web page (which weren’t magnified). The extended demo was great though! The gitpod “sales pitch” was a nice side effect.

[2024 dev2next] Shaping your hiring process

Posted on by
Reply

Full title: Shape Your Hiring Process to Attract & Engage Better Tech Talent

Speaker: Erica Woods

For more see the table of contents

Notes

  • Motivation for talk: 8 candidates declined in a row

Top 10 reasons managers lose good candidates

  • Lack of flexibility/WFH
  • Lengthy process
  • Rigorous process/bad experience/over interviewed
  • Poor engagement
  • Not “sold” (poor marketing)
  • Inaccurate/incomplete job details
  • Unrealistic requirements list
  • “Too many cooks” issue
  • Not enough money
  • Counter offers/better offers

Think about

  • How many steps?
  • How long between?
  • How long total? – ideally 2-3 weeks. Or 1 week for great candidate
  • Where can we lessen?
  • Personal mentality?
  • What else can we do with candidates we like?

Other problems

  • Rescheduling interviews so many times
  • Communicate roadmap if more appealing
  • Titles that don’t have meaning – “I don’t work there; I don’t know what that means” – ex “Solution Engineer 3”. Add “we are looking for someone to act as <common role name>”
  • 456 different BA titles across clients

Position attractors – 10P model. Include some in job description. Can divide and conquer so different interviewers cover different ones. Also good checklist if you are interviewing.

  • Purpose
  • Project
  • Problems – ex: what do in next 6 months
  • Priorities
  • Place – ex: location vs remote and place within organization, size of team
  • People
  • Perks
  • Pay
  • Path/Potential
  • Pain Points

80/20 rule

  • Identify candidates with 80% of the requirements
  • Make rest nice to have
  • Identify growth opportunities
  • Communicate training opportunities

Rapport

  • Chit chat
  • 92-93% interviewees are nervous
  • 2 minute rule start out
  • Other specifics about yourself
  • On video, eye contact (with camera) and show hands
  • Commonality via resume/linked in
  • Understand motivators and align responsibilities and tech stacks
  • Help visualize working there
  • Physical or virtual background as icebreaker
  • Bring up stuff from Linked In – shared connections, info

Candidate scorecard

  • Points given for various skills. Also text areas
  • Ask team what important for all positions with respect to soft skills
  • Differentiate between culture fit vs role/tech skills
  • Share with hiring partners so get better candidates over time
  • In Germany, can’t keep the data unless candidates consent. Not all candidates will give it.

AI

  • Market research (skills, salaries, insights)
  • Job descriptions – ask to make more attractive for role
  • Candidate vetting/skills identification
  • Interview questions
  • Communications (offer or rejection letters)

Other notes

  • Review process annually
  • Remember your interview experience and what care about as candidate. Would you apply for this job?
  • Copilot can help clean up job descriptions. They use good language
  • All apex recruiters are skillset focused.
  • Requirement “what I need; list of skills” vs opportunity “here’s how this will be a fulfilling career move”
  • Candidates now taught to ask AI to learn about person
  • If intimidating because of social media, have recruiter humanize you by telling a story
  • Candidates often felt bombed even when did fine. Managers ask an escalated line of questioning; you aren’t going to know anything

My take

I was a little tired. A few sentences in this little boy walks in. Erica invites him to sit down. He does and then says his mom might worry where he is because he was in the bathroom. Then left. So cute and woke me right up.

Erica warned us several times up front it is a marketing heavy presentation. That was fine; different perspectives are nice. Overall good, I do wish it was targeted more to the audience. Some of the stuff is things developers/architects/teams can’t just change. Similarly, a Project Manager position as the job description example isn’t the best fit for the attendees of this conference. But there were also lots of things that the actual interviewer can do. Audience interactivity was great

[2024 dev2next] improving llm results with rag

Posted on by
Reply

Speaker: Brian Sletten (bsletten@mastodon.social)

For more see the table of contents

PDF of deck on dropbox

Notes

  • Problem: info in our language but models insufficient to extract it
  • Important to capture sequences – ex: context window
  • Problems with word2vec and other embedding approaches. Sequences lost impact if got too long. “New York” and “Soviet Union” useful since near each other. Words farther apart are harder to predict
  • Next transformer architecture used levels of “attention” to have more detailed views between/across sentences
  • Encode in a lower dimensional space and decode into higher dimensional space
  • Positional encoding of words in sentences – picks up some nuance – some has quadratic calculations, but can parallelize so fast
  • Expensive to create a model. Still expensive but less so to tune it
  • Types of RAG: Naive, Advanced, Modular

Emergent behavior

  • Not magical/sentinence
  • Avoids need to have to retrain all the time
  • Use linguistic skills, not knowledge skills
  • Chain of thought prompting

Causes of hallucinations

  • No logic engine/no way to evaluate correctness
  • Language engine with schocastic element to avoid memorizing and encourage novelty

Example

  • Showed how can access web.
  • Allows you to summarize current news stories
  • Note: can include output format in prompt: As Json, As CSV, In bulleted list format, etc

Options

  • Basic model
  • Fine tuning – setting parameters
  • Prompt engineering

RAG

  • Allows getting a lot of custom data
  • Work with vector databases

Searching

  • Find portion of data. Then do kd tree and nearest neighbor search
  • Invevertible tree
  • Hierarchical Navigable Small Worlds (HNSW) – start in high dimensional space then detailed search
  • Like express to local train in a city
  • Can find docs that mention a keyword and then use those docs to answer questions
  • Want to minimize long contexts because costs lots of tokens
  • Chunking makes docs smaller so pay less for search – llama provides API to chunk

Limitations of Naive RAG Models

  • Issues with precision and recall: misaligned chunks, irrelevant or missing chunks
  • Can still hallucinate if no backed by the used chunks
  • Still have toxicity and bias problems

Chaining

  • Initial response
  • Constitutional principal – showed how to add ethics/legality and rewrites
  • Constitutional principal – added rewrite for 7th grader and rewrites
  • That gives final response

Security

  • Easy to poison data
  • Need data cleansing but cleverer
  • http://berryvilleiml.com – machine learning security

Reference: https://www.louisbouchard.ai/top-rag-techniques/

My take

I learned a bunch and the examples were fun. Good to see the code and output. My brain filled up during the session. I needed to switch to an easier talk for the final 5:30 session as I don’t have enough focus left. Cool how the answer to security was a different deck!