Speaker; Micah Silverman @afitnerd)

For more see the table of contents

---

Notes

• ChatGPT took 2 months to get to 100 million global monthly active users. By contrast, TikTok took 9 months, Uber 70 months, Instagram 30 months
• Hot trend, but also people found utility in it.

App Security

• Getting hard.
• Code growing faster and apps getting more complex

Common uses in dev

• Adding comments
• Summarizing Code
• Writing “readme”
• Refacotring code
• Proividing templates
• Pair programming
• Generating code – the new stack overflow

Stats and studies

• 92% using AI coding use
• 57% completed tasks faster (not necessarily better)
• 27% more likely to finish task
• 40% co-pilot code contained vulnerabilities
• More likely to believe wrote more secure code, but wrote less secure code. Because believed was more secure, didn’t look hard.

AI code

• Like junior dev just out of bootcamp. Need to checked works and secure
• Example hallucinations. Change over time. Over a few months, went from 98% on math to 2% on math. Open AI fixed basic math. Designed to be good prediction engines, not math
• “chatGPT is confidently wrong” – Eelko de Vos

AI Coding

• Asked for an Express app to take name in request param and returns a website showing name
• All LLMs tried had XSS/injection
• If questioned or asked to create a secure express app, would get sanitized one. Let of sanitization varies.
• Showed Synyk advisor – gives health score on libraries – ex: sanitizer. Need to check recommended libraries

Co-pilot

• 40% code trained on is insecure
• Used approach where prompt through comments (vs chat feature)
• Used live templates to autocomplete prompt comments to save time for demo
• Example with Spring boot and Thymeleaf. Copilot got that from context of project
• Not quite right but made minor changes vs starting from scratch
• Copyright was 2017; noted hadn’t done that before
• Copilot tried to provide the next comment/prompt. Not what wanted, but reasonable
• Synk IDE extension – Detected SQL injection in view that looks like problems view

Chat GPT

• Had do a security code review
• Added HtmlUtils.htmlEscape(username) – context aware; knew using Spring Boog

My take

Micah said up front that he has no connection to Microsoft or IntelliJ and is just using their products. I never thought to give that disclaimer when I use tools. I’ll think about whether I want to when it isn’t almost 9pm. I am very much a morning person. In fact, that’s why I chose this talk. I thought it would require loading the least info into my mind to understand at this hour while still learning. The demo of copilot for building an app was fun with a good emphasis on security.