Tag Archives: ai

Free Copilot.. Which IDEs shall I use

Posted on by
Reply

Earlier this month, Microsoft announced free GitLab Copilot. You get 2000 autocompletes a month and 50 chat messages. The idea is clearly that you’ll start relying on it and then pay.

I installed and configured in both VS Code and IntelliJ. Setup was easy. I had to authorize the IDE by entering a code on github.com that the IDE gave me. It also gave me the link to enter it so pretty easy.

I also installed it on PyCharm and quickly uninstalled it. Copilot was being too helpful and I need to practice more typing the idioms as I get better at Python. (This is my fifth time learning Python. Every time I get decent at it, I don’t use it for a long time and forget. Maybe this time it’ll take!)

I do use AI periodically to ask how to improve the Python code I write (at home on personal stuff). But I can do that outside my IDE.

I like that Microsoft is doing this. They’ve had a free trial of CoPilot for a long time, but that is time limited. I like that this one is usage limited. So if you don’t use it much, you have it when you need it and still free.

[2024 dev2next] Breaking AI

Posted on by
Reply

Speaker; Micah Silverman @afitnerd)

For more see the table of contents

Notes

  • ChatGPT took 2 months to get to 100 million global monthly active users. By contrast, TikTok took 9 months, Uber 70 months, Instagram 30 months
  • Hot trend, but also people found utility in it.

App Security

  • Getting hard.
  • Code growing faster and apps getting more complex

Common uses in dev

  • Adding comments
  • Summarizing Code
  • Writing “readme”
  • Refacotring code
  • Proividing templates
  • Pair programming
  • Generating code – the new stack overflow

Stats and studies

  • 92% using AI coding use
  • 57% completed tasks faster (not necessarily better)
  • 27% more likely to finish task
  • 40% co-pilot code contained vulnerabilities
  • More likely to believe wrote more secure code, but wrote less secure code. Because believed was more secure, didn’t look hard.

AI code

  • Like junior dev just out of bootcamp. Need to checked works and secure
  • Example hallucinations. Change over time. Over a few months, went from 98% on math to 2% on math. Open AI fixed basic math. Designed to be good prediction engines, not math
  • “chatGPT is confidently wrong” – Eelko de Vos

AI Coding

  • Asked for an Express app to take name in request param and returns a website showing name
  • All LLMs tried had XSS/injection
  • If questioned or asked to create a secure express app, would get sanitized one. Let of sanitization varies.
  • Showed Synyk advisor – gives health score on libraries – ex: sanitizer. Need to check recommended libraries

Co-pilot

  • 40% code trained on is insecure
  • Used approach where prompt through comments (vs chat feature)
  • Used live templates to autocomplete prompt comments to save time for demo
  • Example with Spring boot and Thymeleaf. Copilot got that from context of project
  • Not quite right but made minor changes vs starting from scratch
  • Copyright was 2017; noted hadn’t done that before
  • Copilot tried to provide the next comment/prompt. Not what wanted, but reasonable
  • Synk IDE extension – Detected SQL injection in view that looks like problems view

Chat GPT

  • Had do a security code review
  • Added HtmlUtils.htmlEscape(username) – context aware; knew using Spring Boog

My take

Micah said up front that he has no connection to Microsoft or IntelliJ and is just using their products. I never thought to give that disclaimer when I use tools. I’ll think about whether I want to when it isn’t almost 9pm. I am very much a morning person. In fact, that’s why I chose this talk. I thought it would require loading the least info into my mind to understand at this hour while still learning. The demo of copilot for building an app was fun with a good emphasis on security.

[devnexus 2024] ai proof your career with software architecture

Posted on by
Reply

Speaker: Kelly Morrison

For more, see the 2024 DevNexus Blog Table of Contents

HIstory

  • Fairly recent. GPT created in 2018. Number parameters increasing exponentially
  • Microsoft CoPilot released in 2021. Uses Codex; a specialized model off GPT3 for creating code. Trained on billions of lines of GitHub code and can learn from a local code base
  • Amazon released CodeWhisperer in 2022. Can generate code for 15 languages. Specialized for AWS Code Deployment

Basic Example

  • Asked ChatGPT to write a Java 17 Spring boot rest API for stats in a MongoDB with JUnit 5 tests cases for the most common cases
  • Looks impressive on first pass, but then find problems
  • Hard coded info
  • Used Lombok instead of Java 17 records
  • Code doesn’t compile

Complicated Example

  • Asked ChatGPT to write an entire enterprise app for selling over 10K crafts with a whole bunch of requirements like OpenID, Sarbanes Oxley, etc
  • Didn’t try. Instead came back with a list of things to consider in terms of requirements

What AI can/can’t do

  • Can do “Ground level” work.
  • Still need humans for large orchestratoin – ex: architects
  • Can do more self without junior devs
  • Garbage in, garbage out. Trained on public code in GitHub. Not all good/correct. Some obsolete.
  • Humans better at changing frameworks, working with CSS (does it look nice), major architectural changes, understanding impact of code when requirements change

Hallucinations

  • Doesn’t understand. Asks as mime/mimic/parrot
  • If can’t find answer, will give answer that looks like what you want even if made up. Example where made up up a kubectl option
  • Not enough training data on new languages/technologies. More hallucinations when less training data
  • Mojo created May 2023. Likely to get Python examples if ask for Mojo. However, it is a subset of Python with some extra things

Security Concerns

  • Learns from what you enter so can leak data
  • Almost impossible to remove something in a LLM. ex: passwords, intellectual propery, trade secrets
  • Some companies forbid using these models or require anonymous air gapped use. Translate something innocuous into what actually want

Debugging

  • Can human understand AI generated code well enough to debug
  • GPT and Copilot can sometimes debug code, but have to worry about security

Pushback

  • Law – ChatGPT made up cases
  • Hollywood strike – copying old plots/scripts/characters
  • Unclear if generated output can be copyrighted. For now, not copyrightable but could change.
  • Some software is too important to risk hallucinations 0 ex: plane, car (although Telsa getting there), pacemakers, spacecraft, satellites
  • Lack of context – other software at compnay, standards, reuse, why use certain technologies, securities
  • Lack of creativity – need to determine problem to solve or new approaches

What AI does well

  • Low level code gen (REST APIs, config, database access)
  • Code optimization
  • Greenfield development
  • Generateing docs or tests
  • Basically the kin of tasks you hand off to a junior developer [I disagree that some of these are things you hand off]

Career Advice

  • Focus on architecture, not code
  • Don’t just learn a langauge or framework.
  • Learn which langauges are best in different situations
  • Learn common idioms
  • Look at pricing, availability of libraries and programmers
  • Learn which architectures should be implemented in different languages
  • Learn how to create great prompts for code generation
  • Learn how to understand, follow, test, and debug AI generated code

Book recommendations

  • Building Evolutionary Architectures
  • Domain Driven Design
  • Fundamentals of Software Archicture
  • Head First Software Architecture

More skills

  • Types or architecutures – Layered, event driven, microkernel, microservices, space based, client/server, broker, peer to peer, etc
  • Determine requirements – domain experts don’t know enough about software to specify. Can be bridge between AI and domain experts

Mentoring junior developers

  • Teach how write high quality prompts.
  • Remind to ask for security, test cases, docs, design patterns, OWASP checks
  • Show to spot and deal with hallucinations
  • Help to understand and debut AI written code
  • Help learn architecture by explaining why choices made
  • Ensure code reviews are held
  • Precommit git hooks to test code
  • Use AI to help generate unit tests

ArchUnit

  • archunit.org tests architecuture.
  • Can add own architecture rules.
  • ex: never use Java Util Logging or Joda Time
  • ex: fields should be private/static/final
  • ex: no field injection
  • ex: what layers are allowed to call
  • Can include “Because” reason for each rule
  • Ensures AI doesn’t sneak in something that goes against conventions

My take

Good examples. I was worried about the omission of “where to senior devs” come from but there were examples like changing frameworks so not entirely ignored. Good examples from the ecosystem as well. Good list of skills to focus on.