I asked ChatGPT to explain mutation testing to a five year old to see what it would say. I got this back:

Alright, imagine you have a big puzzle made of Lego blocks. Each block represents a piece of code that makes up a computer program. Now, let’s say we want to make sure that our puzzle is put together correctly so that our program works properly.

Mutation testing is like having a mischievous Lego wizard who comes and changes some of the pieces in our puzzle to see if we notice. He might swap a blue block for a red one, or turn a block upside down. After he’s done, we carefully check our puzzle again to see if it still looks right and if our program still works.

If we can still solve the puzzle and our program still works even with the changes the wizard made, then we know our puzzle – or our program – is really strong. But if we can’t solve the puzzle or our program stops working, then we know there’s a problem and we need to fix it. So, mutation testing helps us make sure our programs are really robust and can handle unexpected changes.

It’s a good example of how ChatGPT is an aide and not a solver. First paragraph is good. In the second part, I love the mischievous Lego wizard who comes and changes some of the pieces to see if we notice. It’s a good premise. We see if our program still works with the changes.

The third paragraph leads us astray. If we notice the change, that is good. If the program stops working it is also good, means we noticed the change. Mutation testing is about having good tests that can detect change. Not a robust program with unnecessary code that behaves the same way when changed.