Why you should never ship code you don't understand
3m read time

Why you should never ship code you don't understand

If you can't explain your code to a colleague without saying 'the AI wrote that', it doesn't belong in your repo. On black boxes, self-validating tests, and why hope is not a strategy.

The temptation is real. You type a prompt and within seconds an AI agent spits out a complex class or algorithm that would normally take you an entire afternoon.

But that's exactly where the problem starts. If you're using an agent for code that you, given enough time and the docs in front of you, couldn't have written yourself, then you're building on quicksand. You're trading fundamental understanding for speed. And in our world, that's a debt you always repay at a punishing interest rate the moment a bug surfaces. That interest is now plainly measurable: code churn, lines rewritten within two weeks.

The black box in your stack ​

The moment an agent writes code that's beyond your own reach, you create a black box that slowly petrifies in your own application. You're missing the insights into why certain decisions were made. Why this data structure? What does this do to your memory when you need to scale?

If you can't reproduce the logic yourself, you can't properly review it. Let alone maintain it. You become a passenger in your own codebase. The moment the AI makes a subtle mistake, you don't have the context to see where things go wrong. You're no longer programming. You're hoping the AI got it right. And hope is not a strategy.

The self-validating test trap ​

The biggest danger lies in your tests. There's a hard rule that many developers forget in their enthusiasm: never let the agent that wrote the code also write the tests. If you do, you get the classic case of the fox guarding the henhouse. If you want AI-written tests anyway, use mutation testing to measure what they're worth.

An LLM works on probability and patterns. If the AI makes a wrong assumption in the code, say by missing an edge case, there's nearly a 100% chance that same mistake ends up in the unit tests. Your tests turn a beautiful green, not because the code is correct, but because the test simply confirms the bug. You're automating your own blindness.

Where it goes wrong in practice ​

A few examples where that AI tunnel vision comes back to bite you:

The off-by-one error: The agent writes a filter but forgets the last element. The test the AI generates also expects that incomplete list. Everything seems to work, until you're missing data in production.

Security: An agent generates a SQL query that's wide open for injection. The test only checks the happy path and sees no problem, because the AI "thinks" it's secure enough.

Business logic: The AI invents a discount rule that runs just fine technically, but completely violates the business rules. Your test confirms the calculation, but your business model no longer holds up.

Take back control ​

A coding agent is a fine assistant, but a terrible architect. Use it for your boilerplate or as a rubber duck, but keep the pen in your own hand when it comes to core logic.

If you can't explain the code to a colleague without saying "the AI wrote that", it doesn't belong in your repo. That sentence has stopped working as a legal defence too. Stay the boss of your own stack.

(2 of 40)
1You don't have an AI problem. You have a process problem.2Why you should never ship code you don't understand3Stop copy-paste engineering4The lava layer: why AI code is slowly petrifying your codebase5The brilliant parrot problem: what AI actually does when it 'thinks'6The prompt is not the spec7The bureaucracy of bots: why we are checking the checker8The day Claude deleted my production database9The arms race for your trust: Mythos, Cyber and the security hype10Stop letting your agents write Markdown11Your agent's suffering is your technical debt speaking12You can't spot the bug if you didn't write the code13One in four: the security debt nobody's counting14Your 10x developer is gated by a 0.1x pipeline15Benchmarks said frontier. Developers said "dumb."16Caveman vs context-mode: small mouth, or smaller room?17Code churn is the lava you can still measure18The ceiling is made of concrete19The token-saver tax: walking back my Caveman advice20Even the malware is AI slop now21ThePrimeagen was right22Tokenmaxxing is what happens when you measure the wrong thing23They just asked the bot nicely: your support agent is the attack surface24Speed got cheap. Judgement didn't.25Your coding agent has no world model. You built it one.26The Ferrari has a limiter: a day with Claude Fable 527The off-switch was never yours28An open MCP server is worse than an open database29The most resilient job is eating its seed corn30The off-switch works both ways now31AI writes the tests. Mutation testing checks if they work.32How to get better at reading code: a practice routine33Learning to program in the age of AI: what I would learn first34Who is responsible for AI-generated code? You are, and 2026 wrote it down35When not to use AI for coding: the tasks I still do by hand36Hiring junior developers in 2026: the collapse is a hiring decision37Software estimation with AI: the typing was never the estimate38Slopsquatting: checking that the package exists is not a defence39Eight reports, one bar: judging an AI safety sprint40Nobody's agent went rogue