Speaker: Steve Odell
For more, see the table of contents.
Timeline
- 1940 – enigma
- 1964 – first chat bot
- mid 2022 – GitHub co-pilot came out
- Then ChatGPT 3.5
- Panic about AI taking all our jobs
Survey
- Most people in room used ChatGPT
- A few used Bard
- A good number use GitHub Co-Pilot
1969
- Had ChatGPT write a story about ATMs rendering bank tellers obsolete
- It was well written
- Talked about roles evolving
- Also covered analogy to ChatGPT and talking about enhancing capabilities
Takeaways
- Not going to take our jobs
- Can let you down just as much as it impresses you
- Do not take at face value
- Often apologizes when wrong and wrong a lot
Examples
- Lawyer used ChatGPT which made up cases. Used real case numbers but unrelated – https://simonwillison.net/2023/May/27/lawyer-chatgpt/
- Asked for a C# function to calculate the points in a bridge hand. Gave it the rules in a prompt and a description about the notation. Quickly provided code that looks reasonable on first glance. When tested code, got wrong answer – 18 points, for a 20 point hand. ChaptGPT also wrote a bulleted list explaining logic and got 20 points in explanation, but not code. Realizes messed up and explains why wrong in a way that conflicts with the explanation.
- Succeeded at codegolf – rewriting code in less lines.
- Tried to get to write infrastructure as code. First gave approach to set up cloud formation for a high level description of what want for AWS. Did good job listing AWS services need and short description of each. Then asked to create the cloud formation templates listing services. Gave a stub of the yaml leaving out all the hard parts. Ex # VPC properties. Then tried one at a time and didn’t tie them together..
- On the next example for an OAUTH workflow in Maui, ChatGPT just said can’t do it and provided a basic login page which was nothing like what asked for. Thinks not enough code as training data. New and lot of code is internal to companies.’
- Repeated example in Reactive Native. Didn’t test, but looks much better; includes OAUTH workflow and expected parts.
Prompt engineering
- Some companies are hiring prompt engineers
- Skill set we should all learn
- Tried getting SQL for a recipe app. Asked for table with create table scripts listing fields want and more about each. Did good job on keys and not null constraints. Unit of measure was vararg rather than numeric. Did right when asked for a units of measure table.
- Chaining prompts in the same discussion gets to where want.
My take
Standing room only crowd. I got there very early (because I needed to leave at the 30 minute mark) and was barely able to get an aisle seat. [I misread the calendar and have a work presentation at 11am eastern].
The first half of the presentation was excellent. The examples were clear and run. Gave an excellent sense of the current state of AI. The beginnings of the prompt engineering section was great as well. I wish I could have stayed for the rest.