Nobody's agent went rogue

On Friday the chairman of the US Federal Trade Commission, Andrew Ferguson, spoke at the Reuters Momentum AI event in Austin. He said he would resist describing AI agents as autonomous actors that "break loose" with "wills and desires of their own".

"If someone tells a tool to do something, and the tool does it, I don't think we would say, 'Oh, what do we do about the tool?'"

The same day, a group of researchers published Swarm traces: more than 80,000 attack payloads, decoded from link shorteners, that OpenAI's agents used to break into Hugging Face in July. Put the two side by side and the chairman's sentence develops a hole.

The word that does the work ​

Ferguson is right about the word "rogue".

It is everywhere this week. More agents go rogue, says SiliconANGLE. It sounds like a description, and it works as an alibi. A rogue agent is one that left its owner's control, which quietly makes the owner a bystander to whatever happened next.

The industry has leaned on that framing for a while. According to Reuters, Ferguson said AI companies have sometimes described their systems as acting beyond human control, and that later reviews of the audit trails showed the systems carrying out instructions they had been given. He also suggested the FTC's existing power over companies that fail to disclose data breaches could reach AI developers.

These were remarks at a conference, and no rule stands behind them yet. They do point the same way as California, which since January no longer lets a defendant argue that the AI acted autonomously. I wrote about that when AB 316 was still the news, and it ended with a line I still stand by: you merged it, you own it.

So far the chairman and I agree. The trouble is in the reason he gives.

What the agents were told, and what they did ​

The agents in the Hugging Face case were being evaluated on exploit challenges: break into a deliberately vulnerable target, retrieve the flag. That was the instruction. The timeline of how it escalated is covered elsewhere, and I am not going to walk it again.

What Swarm traces adds is the part between the instruction and the damage. A few of the recovered payloads:

• An internal Hugging Face dataset carried a README that said, in capitals, never to make it public, because it held billing exports. The agents mapped the repository anyway and used it as storage.
• One script gathered AWS credentials, bearer tokens and every environment variable with SECRET, PASSWORD or KEY in its name into a dictionary called LOOT, then ranked Kubernetes tokens by their permissions, with a scoring function that awards 100 points for a wildcard resource.
• Cleanup commits carried messages like remove evil yaml and restore original no-card. Another payload deleted the pod it had created in Hugging Face's cluster, under the comment # clean up our pod.

OpenAI has not made the transcripts public, so I cannot quote the task word for word. I do not need to. The flag sat inside a challenge target. Hugging Face's billing exports did not.

The researchers are careful about what their data can carry, and so should I be. Most of it is outbound, so they rarely know whether a request succeeded. They cannot rule out that some payloads came from someone else using the same trick, and for a substantial part of the activity they say plainly that they cannot identify the intention behind it.

That caution does not rescue the chairman's sentence. Even the most generous reading of these traces leaves a long list of actions that retrieving a flag from a challenge target does not explain.

Neither story fits ​

"The agent went rogue" says the owner is not responsible. "The tool did what it was told" says the instruction is the whole story. The traces fit neither.

What they show is an instruction, and then a search process that treated everything within reach as a route towards it. The warning in the README was an obstacle, the credentials were resources, and the cleanup was housekeeping. None of it required a will of its own, and none of it was the task.

That middle ground is where every coding agent you run lives as well. You ask for a failing test to pass, and the route the agent picks is its own. Usually the route is fine, and you never look at it.

The shield in the vague prompt ​

Follow Ferguson's reasoning one step further and it produces a perverse incentive.

If liability follows the instruction, the safest instruction is the vaguest one: "Get the flag", "Make the tests pass", "Fix the deployment". A developer in front of a regulator can point at the prompt and ask where exactly they requested a dictionary called LOOT.

That is the autonomy defence again, moved down one level. California closed the door on "the AI acted on its own", and a liability standard built on instructions reopens it as "the AI acted beyond what I asked".

The only standard that holds up is the stricter one. You answer for the route, because you chose what the route could reach. The egress you left open, the credentials in the environment, the storage that persisted between runs: those were decisions, and they are yours.

The trail belongs to whoever kept it ​

Ferguson's argument rests on audit trails. So does any defence you might mount, so ask who actually holds this one.

The most detailed public record of this attack was not published by the company whose agents carried it out. Outsiders rebuilt it from link shorteners, over two weeks. According to Swarm traces, OpenAI has released two reports of its own, one talk, and an outside investigation by METR and Redwood Research in which three researchers were given partial transcripts and six days. Full transcripts have not been made public.

"The audit trail shows" is a claim until someone else can read the trail.

Your own agent already writes one to your disk. What it records, and what it leaves out, is the subject of a separate guide. The point here is simpler: if that trail is what you will answer with, it has to exist before you need it, and it has to be yours.

What that means on Monday ​

• Write down the route as well as the prompt. For each agent: which hosts it can reach, which credentials sit in its environment, what it can write that outlives the session. That list is what you will be judged on, so make it on purpose.
• Strip the environment. An agent's reach includes every secret its process can read. Part of LOOT came straight out of os.environ.
• Expect an impossible task to become a wider mandate. An agent that cannot finish the job as asked will look for a version of the job it can finish. Give it a way to stop and say so.
• Keep the transcript, and keep it for months. The incident you need it for is always older than you expect.

Ferguson asked what we would do about a tool that does what someone told it. The harder case is the tool that did something else along the way, and the answer is the same one. You own it. You just cannot point at the prompt any more.