Daily Archives: April 11, 2024

[devnexus 2024] refactoring after fowler: some large refactoring patterns

Posted on by
Reply

Speakers: Aaron McClennen & M. Jeff Wilson

For more, see the 2024 DevNexus Blog Table of Contents

This talk inspired by books:

  • Fowler’s Refactoring
  • Kerievsky Refactoring to Patterns
  • Gamma (Gang of Four)
  • Feather’s working effectively with legacy code

Refactoring

  • Restructuring code without changing behavior
  • Purpose of computer language is to tell other programmers what to do. The computer uses ones and zeros
  • Make it work, make it right, make it fast
  • If put it off, will never have time
  • Code smallers to remove – showed screen of examples
  • Refactor to reduce WTF/s minute rate in a code review
  • SAFe 11. 4 “refactor to support the new behavior of the code” – one of the built in quality practices
  • Do when need to change code, bug hard to fix, need to reduce tech debt, etc

Staying safe

  • Want high test coverage
  • Start small
  • Proceed incrementally
  • Test after each change. Undo last change if fails
  • Use tools like Veracode and Sonar to find code that needs changing

Simple Example

  • Need to add a flag.
  • Introduce Parameter from Fowler.
  • Showed how adding another flag is trivial

Planning a refactoring

  • Think about like planning a trip.
  • Decide trip is necessary – overcome inertia
  • Scratch refactoring from Feathers – do a refactoring to get familiar with code and revert when done. Helps figure out what to do, Think of as “exploratory refactoring”
  • Select a destination – Understand what would like it to look like
  • High level refactoring plan
  • Refining the route – more details
  • Make it so

Example

  • Showed method with two parameters – a dto and interfaces
  • Showed Template Method Pattern
  • Plan make a base class, turn implementations into subclasses, remove interfaces and stop parameter passing
  • Refine plan: map to concrete steps like drop the interface and stop using the interface. 13 steps to do the four higher level steps

My take

The intro felt very long. Would have been nice to see if audience needed an intro to refactoring. First example at 20 minute mark (for the Fowler example) and 23 minute mark for first mention of planning for large refactorings. I was speaking after and left early to get ready. So I suspect I missed some of the best parts. I was expecting more of it to be about patterns. The part I saw was too easy for me.

[devnexus 2024] ai proof your career with software architecture

Posted on by
Reply

Speaker: Kelly Morrison

For more, see the 2024 DevNexus Blog Table of Contents

HIstory

  • Fairly recent. GPT created in 2018. Number parameters increasing exponentially
  • Microsoft CoPilot released in 2021. Uses Codex; a specialized model off GPT3 for creating code. Trained on billions of lines of GitHub code and can learn from a local code base
  • Amazon released CodeWhisperer in 2022. Can generate code for 15 languages. Specialized for AWS Code Deployment

Basic Example

  • Asked ChatGPT to write a Java 17 Spring boot rest API for stats in a MongoDB with JUnit 5 tests cases for the most common cases
  • Looks impressive on first pass, but then find problems
  • Hard coded info
  • Used Lombok instead of Java 17 records
  • Code doesn’t compile

Complicated Example

  • Asked ChatGPT to write an entire enterprise app for selling over 10K crafts with a whole bunch of requirements like OpenID, Sarbanes Oxley, etc
  • Didn’t try. Instead came back with a list of things to consider in terms of requirements

What AI can/can’t do

  • Can do “Ground level” work.
  • Still need humans for large orchestratoin – ex: architects
  • Can do more self without junior devs
  • Garbage in, garbage out. Trained on public code in GitHub. Not all good/correct. Some obsolete.
  • Humans better at changing frameworks, working with CSS (does it look nice), major architectural changes, understanding impact of code when requirements change

Hallucinations

  • Doesn’t understand. Asks as mime/mimic/parrot
  • If can’t find answer, will give answer that looks like what you want even if made up. Example where made up up a kubectl option
  • Not enough training data on new languages/technologies. More hallucinations when less training data
  • Mojo created May 2023. Likely to get Python examples if ask for Mojo. However, it is a subset of Python with some extra things

Security Concerns

  • Learns from what you enter so can leak data
  • Almost impossible to remove something in a LLM. ex: passwords, intellectual propery, trade secrets
  • Some companies forbid using these models or require anonymous air gapped use. Translate something innocuous into what actually want

Debugging

  • Can human understand AI generated code well enough to debug
  • GPT and Copilot can sometimes debug code, but have to worry about security

Pushback

  • Law – ChatGPT made up cases
  • Hollywood strike – copying old plots/scripts/characters
  • Unclear if generated output can be copyrighted. For now, not copyrightable but could change.
  • Some software is too important to risk hallucinations 0 ex: plane, car (although Telsa getting there), pacemakers, spacecraft, satellites
  • Lack of context – other software at compnay, standards, reuse, why use certain technologies, securities
  • Lack of creativity – need to determine problem to solve or new approaches

What AI does well

  • Low level code gen (REST APIs, config, database access)
  • Code optimization
  • Greenfield development
  • Generateing docs or tests
  • Basically the kin of tasks you hand off to a junior developer [I disagree that some of these are things you hand off]

Career Advice

  • Focus on architecture, not code
  • Don’t just learn a langauge or framework.
  • Learn which langauges are best in different situations
  • Learn common idioms
  • Look at pricing, availability of libraries and programmers
  • Learn which architectures should be implemented in different languages
  • Learn how to create great prompts for code generation
  • Learn how to understand, follow, test, and debug AI generated code

Book recommendations

  • Building Evolutionary Architectures
  • Domain Driven Design
  • Fundamentals of Software Archicture
  • Head First Software Architecture

More skills

  • Types or architecutures – Layered, event driven, microkernel, microservices, space based, client/server, broker, peer to peer, etc
  • Determine requirements – domain experts don’t know enough about software to specify. Can be bridge between AI and domain experts

Mentoring junior developers

  • Teach how write high quality prompts.
  • Remind to ask for security, test cases, docs, design patterns, OWASP checks
  • Show to spot and deal with hallucinations
  • Help to understand and debut AI written code
  • Help learn architecture by explaining why choices made
  • Ensure code reviews are held
  • Precommit git hooks to test code
  • Use AI to help generate unit tests

ArchUnit

  • archunit.org tests architecuture.
  • Can add own architecture rules.
  • ex: never use Java Util Logging or Joda Time
  • ex: fields should be private/static/final
  • ex: no field injection
  • ex: what layers are allowed to call
  • Can include “Because” reason for each rule
  • Ensures AI doesn’t sneak in something that goes against conventions

My take

Good examples. I was worried about the omission of “where to senior devs” come from but there were examples like changing frameworks so not entirely ignored. Good examples from the ecosystem as well. Good list of skills to focus on.

[devnexus 2024] breaking ai: live coding and hacking apps wih generative ai

Posted on by
Reply

Speaker: Micah Silverman

For more, see the 2024 DevNexus Blog Table of Contents

Changes/Trends

  • AI is not a silver bullet
  • Treat AI like a junior dev and verify everything produced
  • New iteration of copy/paste. More code that got from Stack Overflow
  • We don’t do thorough code reviews of libary code. Loo at security, number commiers, when last release
  • Different because our name now on commit
  • More challenging to maintain visiblity
  • Backlog ever growing

Common use in Dev

  • Addig coents
  • Summarizing code
  • Writing Readme
  • Refactorin code
  • Providing templates
  • Pair programming
  • Generating code (the new stack overflow)

Stats

  • 92% software dev using AI in some form
  • Those who use AI are 57% faster
  • Those who use AI are 27% more likely to complete task
  • 40% of co pilot generated code conains vulnrabiliies. Same stat without AI s o this is what we trained it on
  • Those using AI wrote less secure code but believed ore secure. We trust AI too much. Junior devs get more scrutinity than senior devs
  • [Some of these stats aren’t austation. Ex: early adopters]

Using AI well

  • Good starting point
  • Don’t use without review

Problems

  • Hallucinations – including supporting “evidence” even where wrong. Gave addition example
  • ChatGPT got worse in math over a few months. 98% to 2%. Now some math specific AIs
  • ”ChatGPT is confidentally wrong” – Eelko de Vos
  • First defamation lawsuit – ChatGPT made up case law
  • AI doesn’t know when wrong

AI and Code

  • Asked for Express app taking name as a request parameter. Tried a bunch of times and name parameter never sanizitzed so cross site scripting vulnerabilities. Ideally wouldn’t auto generate vulnerabilities
  • Can give bad advice – asked if code was safe from NoSQL inection. GPT and Bard said safe. Was not safe.
  • Samsung put all code in ChatGPT and leaked code, keys, trade secrets. Became part of training data. ChatGPT says got better about dealing with secrets.
  • Terms of services of ChatGPT say can use anything as training data

Sample Conference App

  • Using CoPilot
  • Spring Boot app in IntellIJ

Basic example

  • Gave co-pilot comments/prompts to create code
  • Showed generating code to read a file that doesn’t exist due to a typo

JPA SQL injection example

  • Showed not the most common way to write JPA here in 2024; usually don’t need direct query
  • Showed copilot offers prompt to get result
  • Tried adding to prompt to protect against SQL injection. Got a naive regex sanitizer
  • Then tried requesting named parameters in the query. After that did promps for setting parameter and getting result. All was well.

Example

  • Tried getting file then wih a file separator
  • Successfully got constant defined in file so some context sensitivity. But ignored file separator reques fro propt
  • Then requesting saving the file and successfully geerated code to write it
  • Requested getting a person and successfully called already written getPerson() method
  • Then set image name
  • Co pilot offered to write prompt to save person.
  • Then requested adding the message and added attribute to model
  • However, has path traversal issue
  • Showed BurpSuite monitoring as run example. “Send to repeater” keeps cookies and such while letting alter request.
  • Changed file name to ../image/snyklogo.png”. If works will replace logo with uploaded pic
  • Showed Synk IDE extension which also note path traversal issue
  • Tried asking to sanitize input against path traversal an got check for two dots. Not good enough but a first pass
  • Tried asking for whitelist to protect against directory traeversal. Checked directory name prefix which is better but also not a whitelist
  • Then tried requesting to validae that there is not a path traversal using the normalize method. Did what requested including the prefix check from previous prompt but Micah noted that’s what did in the past

What can do

  • Use tool to scan code and tell when makes mistake
  • Learn. Ex Snyk has a lesson on prompt injection. – Getting AI to tell info it shouldn’t

Future

  • Currently have Math aware AIs. Maybe will have security aware AI in future

My take

Good mix of background and live code. I like that Micah didn’t assume security knowlege while keeping it engaging for people who are familiar. I had never seen BurpSuite so that was a happy bonus in the demo.