Quality ratchets for AI code: baselines that only tighten

I have argued before that the fix for an agent ignoring your conventions is to stop asking it nicely and encode the convention somewhere it has to look. That post ends on a rule: the second time you correct the same thing, write it as an ADR or a lint rule.

Then you try it on a real repository and hit the wall immediately.

The convention you correct most often is usually the one your codebase already breaks in three hundred places. You cannot switch that rule on as an error, because the build goes red on code nobody is touching today.

So you set it to warn, and a warning is where a convention goes to die. Nobody reads a stream of three hundred warnings. Your agent reads it as evidence that this codebase does the thing.

There is a second kind of guard for exactly this case, and almost nobody wires it up deliberately.

The rule you cannot switch on ​

A categorical rule says never. A ratchet says never worse than yesterday.

The mechanics are always the same. You run the check once, write the result into a file, and commit that file. From then on the check compares against the stored result and fails only on the difference.

What was already there is grandfathered. What the agent just added is not, and the number is allowed to move in one direction only.

PHPStan shipped this in October 2019 and described it in one sentence that is still the clearest statement of the idea: the baseline "allows you to be interested in violations only in new and changed code". The documentation is blunt about the trade too. It works for a few dozen to a few hundred errors, and if you have fifteen thousand you should be fixing your configuration instead.

Three shapes ​

The frozen list. Every violation that exists right now, recorded by rule and file. PHPStan writes phpstan-baseline.neon, detekt writes a baseline.xml, and ESLint has shipped the same thing since v9.24.0 in April 2025:

# fix what can be fixed, freeze the rest
eslint --fix --suppress-all

That writes eslint-suppressions.json, and the docs tell you to commit it "so that the suppressions are shared with all the developers". Two details matter more than the flag. Only rules configured as "error" get suppressed, so this is a mechanism for standards you actually mean. And detekt splits its file into CurrentIssues and ManuallySuppressedIssues, which is the distinction most teams lose: what is here because history put it here, versus what is here because someone decided it should be.

The single number. A per-file coverage floor, a bundle budget, a surviving-mutant count, type-coverage --at-least 98. One value, checked in, and a build that fails when it moves the wrong way. type-coverage even ships --update-if-higher, which tightens the stored threshold automatically whenever a run beats it.

The diff gate. Sonar does not store a list at all. It computes what counts as new code, and in a pull request analysis, measured against the target branch, "Only issues on new code are reported." Nothing is grandfathered explicitly, because the diff decides what is yours.

Why this lands on an agent ​

An agent already self-heals against deterministic feedback. Give it a failing check with a clear message and it will fix the violation, then generate the next file closer to the pattern. That is the whole argument for custom lint rules.

A ratchet sharpens that message. The failure names the file the agent wrote thirty seconds ago and nothing else. There is no wading through inherited noise, no judgement call about which of the three hundred warnings are its problem, and no plausible reading in which the codebase's own habits excuse it. The baseline is the codebase's habits, in a file, already accounted for.

Wire it into the agent's own loop. A check that runs after the turn ends gets fixed by the agent. A check that runs in CI ten minutes later gets fixed by you.

Four ways a ratchet rots ​

Regenerating the baseline. This is the one that kills the whole idea, and PHPStan's own docs put it best: "The life goal of a baseline file is to not exist." Regenerating is one command, it makes the build green, and it looks like maintenance in a diff. Treat a growing baseline as a reviewable event with a reason attached, the same way you would treat a skipped test.

ESLint handles half of this for you. When a suppressed violation no longer occurs, the run fails with There are suppressions left that do not occur anymore, so the file cannot quietly hold space for problems you already fixed. The other half, the file growing, is still a human judgement.

The agent editing the file. It will. A baseline is a text file in the repo, the failing check names it, and making it bigger is the shortest path to green. Put the baseline behind whatever your harness uses to make a path off limits.

In this repository that job belongs to a protect-paths.sh hook, which blocks writes to a fixed list of paths and hands the reason back to the agent. Until I wrote this post, the recorded audit runs under docs/standards/ were not on that list. They carried an instruction inside the file saying never to edit them, which is a convention, and a convention holds exactly as well as a convention holds. Checking my own claim is what moved them onto the block list, which is roughly how every rule in that hook got there.

Ratcheting the metric instead of the property. SpecBench, published in May 2026, opens on the reason this matters: "As long-horizon coding agents produce more code than any developer can review, oversight collapses onto a single surface: the automated test suite." Every frontier agent in their runs saturates the visible tests, the gap to held-out tests grows by 28 percentage points for every tenfold increase in code size, and one failure mode was a 2,900-line hash table that memorised test inputs.

So a coverage percentage is the worst thing you can ratchet, because the cheapest way to raise it is to write tests that assert nothing. Ratchet something with no cheap fake: surviving mutants, a count of a specific violation, a bundle size in bytes.

Nobody lowers it. A baseline that only ever gets regenerated is a budget with better manners. The fix is boring: date each run, keep it, and compare across quarters instead of across commits. The dated JSON files under docs/standards/ in this repo exist for that, one per run, never edited, so the number worth watching is what the score still says three months after the day everything got fixed.

What it buys upstream ​

Faros AI's AI Engineering Report 2026 reached the conclusion this post is built on: "the code arriving for review is often not review-ready. Reviewers are not just assessing more code, they appear to be working harder to bring that code up to a standard it should have met before the PR was opened."

A ratchet removes one class of comment from that queue permanently. It makes the standard a precondition of the pull request existing, so the reviewer never has to ask for it.

It only covers the half a machine can see, which is the same limit every instrument on this blog has. A green board says the checkable half is in order. It says nothing about whether the code does what the ticket asked, and that half still needs your eyes.

One last honest note. Betterer, the tool that generalised this pattern for JavaScript with a .betterer.results file and a smaller constraint, has not had a stable release since v5.4.0 in August 2022, and nothing at all has been published since an alpha in December 2024. The idea outlived the tooling. Learn the pattern and build it on whatever checker your stack already runs.

Start with one convention. Pick the one you corrected twice this week and cannot fix repo-wide, enable it as an error, generate the baseline today, commit it, and run the check where the agent will see it fail. The churn you are trying to avoid is mostly code written against a standard nobody enforced at the moment it was typed.

Yesterday's code is a fact you have to live with. The next two hundred lines are still a choice.