Tag Archives: ai

[2024 dev2next] Breaking AI

Posted on by
Reply

Speaker; Micah Silverman @afitnerd)

For more see the table of contents

Notes

  • ChatGPT took 2 months to get to 100 million global monthly active users. By contrast, TikTok took 9 months, Uber 70 months, Instagram 30 months
  • Hot trend, but also people found utility in it.

App Security

  • Getting hard.
  • Code growing faster and apps getting more complex

Common uses in dev

  • Adding comments
  • Summarizing Code
  • Writing “readme”
  • Refacotring code
  • Proividing templates
  • Pair programming
  • Generating code – the new stack overflow

Stats and studies

  • 92% using AI coding use
  • 57% completed tasks faster (not necessarily better)
  • 27% more likely to finish task
  • 40% co-pilot code contained vulnerabilities
  • More likely to believe wrote more secure code, but wrote less secure code. Because believed was more secure, didn’t look hard.

AI code

  • Like junior dev just out of bootcamp. Need to checked works and secure
  • Example hallucinations. Change over time. Over a few months, went from 98% on math to 2% on math. Open AI fixed basic math. Designed to be good prediction engines, not math
  • “chatGPT is confidently wrong” – Eelko de Vos

AI Coding

  • Asked for an Express app to take name in request param and returns a website showing name
  • All LLMs tried had XSS/injection
  • If questioned or asked to create a secure express app, would get sanitized one. Let of sanitization varies.
  • Showed Synyk advisor – gives health score on libraries – ex: sanitizer. Need to check recommended libraries

Co-pilot

  • 40% code trained on is insecure
  • Used approach where prompt through comments (vs chat feature)
  • Used live templates to autocomplete prompt comments to save time for demo
  • Example with Spring boot and Thymeleaf. Copilot got that from context of project
  • Not quite right but made minor changes vs starting from scratch
  • Copyright was 2017; noted hadn’t done that before
  • Copilot tried to provide the next comment/prompt. Not what wanted, but reasonable
  • Synk IDE extension – Detected SQL injection in view that looks like problems view

Chat GPT

  • Had do a security code review
  • Added HtmlUtils.htmlEscape(username) – context aware; knew using Spring Boog

My take

Micah said up front that he has no connection to Microsoft or IntelliJ and is just using their products. I never thought to give that disclaimer when I use tools. I’ll think about whether I want to when it isn’t almost 9pm. I am very much a morning person. In fact, that’s why I chose this talk. I thought it would require loading the least info into my mind to understand at this hour while still learning. The demo of copilot for building an app was fun with a good emphasis on security.

[devnexus 2024] ai proof your career with software architecture

Posted on by
Reply

Speaker: Kelly Morrison

For more, see the 2024 DevNexus Blog Table of Contents

HIstory

  • Fairly recent. GPT created in 2018. Number parameters increasing exponentially
  • Microsoft CoPilot released in 2021. Uses Codex; a specialized model off GPT3 for creating code. Trained on billions of lines of GitHub code and can learn from a local code base
  • Amazon released CodeWhisperer in 2022. Can generate code for 15 languages. Specialized for AWS Code Deployment

Basic Example

  • Asked ChatGPT to write a Java 17 Spring boot rest API for stats in a MongoDB with JUnit 5 tests cases for the most common cases
  • Looks impressive on first pass, but then find problems
  • Hard coded info
  • Used Lombok instead of Java 17 records
  • Code doesn’t compile

Complicated Example

  • Asked ChatGPT to write an entire enterprise app for selling over 10K crafts with a whole bunch of requirements like OpenID, Sarbanes Oxley, etc
  • Didn’t try. Instead came back with a list of things to consider in terms of requirements

What AI can/can’t do

  • Can do “Ground level” work.
  • Still need humans for large orchestratoin – ex: architects
  • Can do more self without junior devs
  • Garbage in, garbage out. Trained on public code in GitHub. Not all good/correct. Some obsolete.
  • Humans better at changing frameworks, working with CSS (does it look nice), major architectural changes, understanding impact of code when requirements change

Hallucinations

  • Doesn’t understand. Asks as mime/mimic/parrot
  • If can’t find answer, will give answer that looks like what you want even if made up. Example where made up up a kubectl option
  • Not enough training data on new languages/technologies. More hallucinations when less training data
  • Mojo created May 2023. Likely to get Python examples if ask for Mojo. However, it is a subset of Python with some extra things

Security Concerns

  • Learns from what you enter so can leak data
  • Almost impossible to remove something in a LLM. ex: passwords, intellectual propery, trade secrets
  • Some companies forbid using these models or require anonymous air gapped use. Translate something innocuous into what actually want

Debugging

  • Can human understand AI generated code well enough to debug
  • GPT and Copilot can sometimes debug code, but have to worry about security

Pushback

  • Law – ChatGPT made up cases
  • Hollywood strike – copying old plots/scripts/characters
  • Unclear if generated output can be copyrighted. For now, not copyrightable but could change.
  • Some software is too important to risk hallucinations 0 ex: plane, car (although Telsa getting there), pacemakers, spacecraft, satellites
  • Lack of context – other software at compnay, standards, reuse, why use certain technologies, securities
  • Lack of creativity – need to determine problem to solve or new approaches

What AI does well

  • Low level code gen (REST APIs, config, database access)
  • Code optimization
  • Greenfield development
  • Generateing docs or tests
  • Basically the kin of tasks you hand off to a junior developer [I disagree that some of these are things you hand off]

Career Advice

  • Focus on architecture, not code
  • Don’t just learn a langauge or framework.
  • Learn which langauges are best in different situations
  • Learn common idioms
  • Look at pricing, availability of libraries and programmers
  • Learn which architectures should be implemented in different languages
  • Learn how to create great prompts for code generation
  • Learn how to understand, follow, test, and debug AI generated code

Book recommendations

  • Building Evolutionary Architectures
  • Domain Driven Design
  • Fundamentals of Software Archicture
  • Head First Software Architecture

More skills

  • Types or architecutures – Layered, event driven, microkernel, microservices, space based, client/server, broker, peer to peer, etc
  • Determine requirements – domain experts don’t know enough about software to specify. Can be bridge between AI and domain experts

Mentoring junior developers

  • Teach how write high quality prompts.
  • Remind to ask for security, test cases, docs, design patterns, OWASP checks
  • Show to spot and deal with hallucinations
  • Help to understand and debut AI written code
  • Help learn architecture by explaining why choices made
  • Ensure code reviews are held
  • Precommit git hooks to test code
  • Use AI to help generate unit tests

ArchUnit

  • archunit.org tests architecuture.
  • Can add own architecture rules.
  • ex: never use Java Util Logging or Joda Time
  • ex: fields should be private/static/final
  • ex: no field injection
  • ex: what layers are allowed to call
  • Can include “Because” reason for each rule
  • Ensures AI doesn’t sneak in something that goes against conventions

My take

Good examples. I was worried about the omission of “where to senior devs” come from but there were examples like changing frameworks so not entirely ignored. Good examples from the ecosystem as well. Good list of skills to focus on.

[devnexus 2024] breaking ai: live coding and hacking apps wih generative ai

Posted on by
Reply

Speaker: Micah Silverman

For more, see the 2024 DevNexus Blog Table of Contents

Changes/Trends

  • AI is not a silver bullet
  • Treat AI like a junior dev and verify everything produced
  • New iteration of copy/paste. More code that got from Stack Overflow
  • We don’t do thorough code reviews of libary code. Loo at security, number commiers, when last release
  • Different because our name now on commit
  • More challenging to maintain visiblity
  • Backlog ever growing

Common use in Dev

  • Addig coents
  • Summarizing code
  • Writing Readme
  • Refactorin code
  • Providing templates
  • Pair programming
  • Generating code (the new stack overflow)

Stats

  • 92% software dev using AI in some form
  • Those who use AI are 57% faster
  • Those who use AI are 27% more likely to complete task
  • 40% of co pilot generated code conains vulnrabiliies. Same stat without AI s o this is what we trained it on
  • Those using AI wrote less secure code but believed ore secure. We trust AI too much. Junior devs get more scrutinity than senior devs
  • [Some of these stats aren’t austation. Ex: early adopters]

Using AI well

  • Good starting point
  • Don’t use without review

Problems

  • Hallucinations – including supporting “evidence” even where wrong. Gave addition example
  • ChatGPT got worse in math over a few months. 98% to 2%. Now some math specific AIs
  • ”ChatGPT is confidentally wrong” – Eelko de Vos
  • First defamation lawsuit – ChatGPT made up case law
  • AI doesn’t know when wrong

AI and Code

  • Asked for Express app taking name as a request parameter. Tried a bunch of times and name parameter never sanizitzed so cross site scripting vulnerabilities. Ideally wouldn’t auto generate vulnerabilities
  • Can give bad advice – asked if code was safe from NoSQL inection. GPT and Bard said safe. Was not safe.
  • Samsung put all code in ChatGPT and leaked code, keys, trade secrets. Became part of training data. ChatGPT says got better about dealing with secrets.
  • Terms of services of ChatGPT say can use anything as training data

Sample Conference App

  • Using CoPilot
  • Spring Boot app in IntellIJ

Basic example

  • Gave co-pilot comments/prompts to create code
  • Showed generating code to read a file that doesn’t exist due to a typo

JPA SQL injection example

  • Showed not the most common way to write JPA here in 2024; usually don’t need direct query
  • Showed copilot offers prompt to get result
  • Tried adding to prompt to protect against SQL injection. Got a naive regex sanitizer
  • Then tried requesting named parameters in the query. After that did promps for setting parameter and getting result. All was well.

Example

  • Tried getting file then wih a file separator
  • Successfully got constant defined in file so some context sensitivity. But ignored file separator reques fro propt
  • Then requesting saving the file and successfully geerated code to write it
  • Requested getting a person and successfully called already written getPerson() method
  • Then set image name
  • Co pilot offered to write prompt to save person.
  • Then requested adding the message and added attribute to model
  • However, has path traversal issue
  • Showed BurpSuite monitoring as run example. “Send to repeater” keeps cookies and such while letting alter request.
  • Changed file name to ../image/snyklogo.png”. If works will replace logo with uploaded pic
  • Showed Synk IDE extension which also note path traversal issue
  • Tried asking to sanitize input against path traversal an got check for two dots. Not good enough but a first pass
  • Tried asking for whitelist to protect against directory traeversal. Checked directory name prefix which is better but also not a whitelist
  • Then tried requesting to validae that there is not a path traversal using the normalize method. Did what requested including the prefix check from previous prompt but Micah noted that’s what did in the past

What can do

  • Use tool to scan code and tell when makes mistake
  • Learn. Ex Snyk has a lesson on prompt injection. – Getting AI to tell info it shouldn’t

Future

  • Currently have Math aware AIs. Maybe will have security aware AI in future

My take

Good mix of background and live code. I like that Micah didn’t assume security knowlege while keeping it engaging for people who are familiar. I had never seen BurpSuite so that was a happy bonus in the demo.