On 10 March, the Financial Times reported from an internal Amazon briefing note describing a "trend of incidents" on Amazon.com, characterised by a "high blast radius" and "Gen-AI assisted changes".
Amazon published a correction on its own site the next day. "In fact, only one of the recent incidents involved AI tools in any way, and in that case the cause was unrelated to AI and instead our systems allowed an engineering team user error to have broader impact than it should have." The one that did involve AI tooling "related to an engineer following inaccurate advice that an AI tool inferred from an outdated internal wiki, and none involved AI-written code."
Two parties with access to internal documents, describing the same week, disagreeing about whether a model was involved. And it does not resolve tidily from there, because the FT went on to amend some of its own assertions, which Amazon's correction notes in passing.
CNBC saw the document too, and reports one more thing: it "originally said generative AI-assisted production changes were partly to blame for the issues, but the reference to GenAI was subsequently deleted." The record of what happened was being edited while people were still arguing over what it said.
The Register had already put the question to Amazon on the day the FT story ran. It got back the line the company had taken in February over a separate incident, that it had seen no compelling evidence that incidents are more common with AI tools, and attached one observation: "the company has provided no data that would allow an independent analysis of incident causes."
That is the whole problem, and it is worth sitting with before reaching for a checklist.
Notice that the two sides are not even disputing the same claim. The briefing note says "Gen-AI assisted changes". The correction says "AI-written code". Those are different things, and both statements can be true at once. Nobody had agreed what the term meant before they needed it.
If Amazon, holding full telemetry over its own retail stack, ends up arguing in public about whether an agent was involved, your reconstruction at 3am is not going to settle it either.
This is a different page than the agent going wrong
A month ago I wrote a runbook for the agent misbehaving: freeze the session, reconstruct from the transcript, work out the blast radius, rotate. That runbook works because the evidence exists. There is a session, a .jsonl transcript, pre-edit file snapshots.
Production is the other case, and it is harder in one specific way.
When the pager fires for code a model wrote three weeks ago, the step every incident process depends on has no addressee. "Ask the person who wrote it" assumes a person who wrote it, holding context that never made it into the diff: what they tried first, what they knew was fragile, which edge case they decided was fine.
Nobody holds that context. It only ever existed inside a session that has since ended.
This post also starts one step late, at the moment you already know something is wrong. What tells you that in the first place is a different instrument and got its own post.
The agent is a second reader, not a witness
The instinct at 3am is to paste the stack trace back into the tool that produced the code.
Do it, by all means. It is a fast second opinion and often a good one. Just be clear about what you are getting, because the framing decides how much weight the answer can carry.
The model has no memory of writing that code. It is reading it now, for the first time, the same way you are, minus everything you know about what production has been doing for the last twenty minutes, plus a strong disposition to produce a confident, helpful-sounding cause. Ask why the code did what it did and you get a plausible reading of the code, generated on the spot, presented in the register of a recollection.
Treat it as a colleague you handed a file to, and weigh the answer accordingly.
Worse, the code has changed since. Fault-localisation research finds this is exactly where models turn brittle. Across 750,013 fault-localisation tasks on ten models, semantic-preserving changes, the ordinary sediment of a codebase that has been touched a few times, "cause LLMs to fail on previously localized faults in 78% of cases".
Four things that have to exist beforehand
None of these are new. Each one costs more when it fails now, because the fallback they all quietly depended on, a human who remembers, is gone.
Ownership, recorded somewhere a machine can read. In a file, because right now it lives in the fact that Sarah knows that service. CODEOWNERS is the cheapest version and it survives Sarah leaving.
A rollback that the database cannot veto. Reverting the deploy undoes the code. It does not undo a migration that has already run, and it does not un-send the emails. This is why the flag is the real undo, and why a schema change is the special case in every workflow.
A runbook per failure class, linked from the alert. The Google SRE book's troubleshooting chapter describes the shape better than any tooling pitch: the alerting system files a bug "with links to the black-box prober's recent results and to the playbook entry for this alert". The runbook is a link the alert carries.
Provenance you can actually query. This is the one everyone gets wrong, and it deserves its own section.
Why the git trailer is not the answer
The obvious place to look is the commit. Most agent tooling can write a Co-Authored-By trailer naming the model that helped, and switching it on feels like solving provenance in an afternoon.
It does not, for three reasons.
It measures configuration, not authorship. A trailer appears when the tool was set to write one. That is a fact about a settings file in a given month, not about who wrote the code. Turn it on in spring and off in autumn and the history says the model stopped working on your codebase.
It sits at the wrong granularity. It is one flag on an entire commit, often a squashed changeset touching a dozen files. It cannot tell you which of those files the model produced, let alone which lines.
It does not survive maintenance. Rename the file, extract the function, reformat the module, and the lines move while the trailer stays behind on a commit nobody will ever read again.
Git records who committed. On-call needs to know how a line came to exist, and those have never been the same question. You merged it, you own it is the right rule, and accountability is all it assigns.
The honest fix sits outside the repository altogether: the deploy record, the review record, the session archive. Which is a records problem, decided long before the pager fires.
What a usable incident record contains
Here the safety world has done work worth stealing, and the first thing to steal is the admission.
The AI Incident Database opens on aviation, which "owes much of its increasing safety to systematically analyzing and archiving past accidents and incidents within a shared database", and where the definitions are clean: an accident involves substantial damage or loss of life, an incident is a case "where the risk of an accident substantially increases". Then it concedes it cannot draw that line for AI, and instead uses "an adaptive criteria for ingesting 'incidents' where reports are accepted or rejected on the basis of a growing rule set".
The most sustained attempt to define an AI incident gave up on the definition and kept the rule set. Do the same. Do not try to settle what counts as an "AI incident" for your team in the abstract, because that is the argument Amazon and the FT were having. Enumerate your failure classes instead, and keep adding to the list.
Two structures are worth copying outright.
The AIID's incident response asks four things: what happened, why it happened, what has been done to remediate it, and what prevents recurrence. That is your postmortem template and it fits in a text file.
The OECD's AI Incidents and Hazards Monitor files each event along several independent facets instead of collapsing it to one number. The ones you can filter on are harm type, affected stakeholders, business function, the task the AI system was performing, and the system's autonomy. The one to import is autonomy. "The agent did it" and "an engineer accepted a suggestion" are different incidents with different fixes, and Amazon's one AI-involved case was the second kind: a human acting on advice a tool drew from a stale wiki.
The part about Article 73, stated carefully
Article 73 of the EU AI Act is the reporting duty everyone cites, and most of what is written about it is either too broad or out of date.
It obliges providers of high-risk AI systems placed on the Union market to report serious incidents to market surveillance authorities, or to the AI Office for the systems it now supervises. Deployers are not exempt, either. Every deadline runs from the moment "the provider or, where applicable, the deployer" becomes aware, and Article 26(5) applies Article 73 to a deployer mutatis mutandis when the provider cannot be reached.
None of which reaches you. Shipping a web application that an agent helped write makes you neither of those things, and your CRUD app is not an AI system to begin with. Anyone implying the duty is yours is selling something.
The dates have also moved, and the coverage has not caught up. The Digital Omnibus on AI, Regulation (EU) 2026/1744, was published in the Official Journal on 24 July and entered into force on 27 July, days before the deadline it changed. It pushed Chapter III, Sections 1 to 3, to 2 December 2027 for Annex III systems and 2 August 2028 for Annex I, citing "the delayed availability of standards, common specifications, and alternative guidance and the delayed establishment of national competent authorities". Article 73 sits in Chapter IX and kept the general date of 2 August 2026.
The quieter half is in Article 111. The AI Act already excused high-risk systems that were on the market before the deadline from most of the Regulation, unless they later underwent "significant changes in their designs". What the Omnibus changed is the date that sentence points at: no longer a fixed 2 August 2026, but whenever Chapter III now applies. The escape hatch was already there, and the window through it got another sixteen to twenty-four months depending on the annex. The prohibited practices in Article 5 still bite, and anything aimed at public authorities has to comply by 2 August 2030 regardless.
So the reporting duty is live while most of the population it would bite on has been carved out or deferred, which is not what "in force from August 2026" conveys on its own.
What is worth taking from it, whether or not it applies to you, is the shape. Reports are due immediately, with backstops behind that: 15 days from becoming aware, two days for a widespread infringement, 10 days where someone has died. Severity sets the clock, which is the correct instinct and the opposite of a uniform SLA.
And the trigger is the sharp bit. The clock starts once the provider "has established a causal link between the AI system and the serious incident or the reasonable likelihood of such a link".
A causal link. Written by people who assumed the records to establish one would exist.
That is the sentence to hold onto, because it is the one thing Amazon, the FT, and you at 3am all lack. You cannot fill in a field you never recorded, and a deadline measured from the moment you establish causation is, for most teams, a deadline that never starts.
What to record, starting with what you already run
None of this needs a platform. It needs four fields written somewhere durable at the moment of deploy, while there is still something to write them from.
- What shipped: the SHA, the deploy timestamp, and who or what approved it. Most deploy scripts already know the SHA and throw the rest away.
- How it was produced: the autonomy level, in one word. Written by a human, suggested and accepted, or authored by an agent under review.
- Who is accountable now, as opposed to who committed it.
CODEOWNERS, and it should be a person. - What was known to be shaky: the thing the reviewer waved through. Review is already the queue everything else is stuck behind, so this one has to cost almost nothing: one line, written at merge time. It is the field that would have caught the Amazon case, because somebody knew that wiki page was old.
The last one is unpopular because it requires admitting something at merge time. It is also the only field on the list that carries the context which used to live in a colleague's head.
If you already have all four, you are unusual. The reason the gap persists is not ignorance. Every one of these controls costs something on an ordinary Tuesday and pays out on a night you have not had yet.
Lightrun's 2026 report, a survey of 200 senior site-reliability and DevOps leaders, found 43% of AI-generated code changes needed manual debugging in production even after passing QA and staging, and that 88% of teams needed two to three redeploy cycles before they could verify an AI-suggested fix. They sell production debugging tooling, so read the framing with that in mind. The 43% is still a number about exactly those nights.
The detail I keep returning to is Amazon's own account of the one incident where a tool was genuinely involved: an engineer, following advice a model inferred from an internal wiki that was out of date. No rogue agent. A stale document, read confidently, acted on in good faith.
That failure mode has been documented on this site as a thing that happens to agents. It turns out it happens to the humans reading what the agent inferred, and it reaches production the same way.