AI and code quality | Blog

~/blog/guide/ai-quality

AI and code quality

Generation got cheap. Judgement did not. On tests, churn and the bar that stays where it was.

What this guide covers

Generating code now takes seconds. Judging whether it is correct, whether it fits the system, and whether it will hold up costs exactly what it always did. That gap is the subject of this cluster, because nearly every quality problem with AI code turns out to be a variant of it.

I write this from practice. I use agents daily and they produce real work. Which is exactly why the friction stands out: output grows, review capacity does not, and everything you skip judging comes back later with interest. Speed got cheap, judgement didn't works that through on what large-scale orchestration does to your review.

It starts before the code exists

Most bad AI code traces back to a bad brief. The prompt is not the spec is the whole argument: a vague intention is not requirements, and the model will happily fill the gaps with something plausible. The sequel is the spec you didn't read, where letting an agent write the spec is fine and forwarding it unread to a second agent is where it goes wrong.

There is also a signal in how hard the agent is finding it. Your agent's suffering is your technical debt speaking: when a model keeps stumbling in one corner of your codebase, that corner was already confusing before it arrived.

Tests that prove something

The obvious reflex is to have the AI write the tests too, and let quality take care of itself. Almost. Coverage only proves a line was executed, never that anyone would notice it being wrong. And a model that makes a reasoning error in the code will cheerfully repeat that error in the test. In AI writes the tests, mutation testing checks if they work I use mutation testing to make that false confidence measurable: sabotage the code and see whether a single test goes red.

Review is the other gate, and it changes when the author is a model. How to review a pull request an AI wrote is what I look for, in what order, and which of your old review instincts stop firing.

What happens to your codebase

Quality reaches past the individual PR. At codebase level three patterns keep showing up.

The first you can measure: code churn, lines rewritten within two weeks because the first version was never really finished. The second is quieter: the lava layer, code nobody understands anymore and therefore nobody dares touch. The third is the one people laugh at until they grep for it: twenty ways to format money in one repo, because an agent with no memory of your helpers writes a fresh one every time.

Measuring AI code quality is how you put numbers on all three, and why three of the obvious instruments hand you a headline figure that flatters AI code instead of catching it. When the diagnosis is in, the consolidation pass is the treatment. Those same instruments get asked a neighbouring question they cannot answer on the spot, which is whether the model you just swapped in is the reason things feel worse. Comparing coding models without a benchmark is the harness for that one, and the reason your instinct to re-run the prompt and diff it proves nothing.

The bar that stays where it was

Which is why this cluster keeps landing on the same standard. Never ship code you don't understand: if you cannot explain a piece of code to a colleague without pointing at the AI, it does not belong in your repo. And keep your reading muscles trained, because you can't spot the bug if you didn't write the code. Debugging instinct is built from thousands of rounds of writing and reading your own work, and it evaporates faster than you expect.

None of these rules are new. They held when code was typed by hand. AI made them more urgent, because the volume at which you can break them has exploded. Typing speed used to be a natural cap on how much poorly understood code could land in your repo per week. That brake is gone and nothing automatic took its place. The replacement has to come from your process, which is what the posts in this cluster try to supply.

Where this touches the workflow

Quality starts upstream, in how you drive an agent. A proper spec, small steps and review as a fixed gate prevent more misery than any linter. The full working method lives in the agentic coding guide, under related topics below.

Below are three starting points, then every post in this cluster, newest first.