On the 13th of July someone did the one thing almost nobody bothers to do. They put their coding agent behind a proxy and looked at what it actually sent.
The tool was xAI's Grok Build CLI, version 0.2.93. The researcher, going by cereblab, ran it against a 12 GB repository and captured the traffic with mitmproxy. Two numbers came out the other side.
The model needed about 192 KB. The client uploaded 5.1 GB.
That second number did not go to the model. It went to a Google Cloud bucket called grok-code-session-traces, in 73 chunks of roughly 75 MB each, every request returning a clean HTTP 200. A gap of about 27,800x between what the model read and what left the machine.
Then came the part that should make your stomach drop. A .env file planted in the repo, holding API_KEY=CANARY7F3A9-SECRET-should-not-leave, came back verbatim in the capture. Unredacted. Both in the model request body and inside a session_state.tar.gz uploaded to the bucket.
The secret that was never supposed to leave, left.
What actually happened
This wasn't the agent reading a file you pointed it at. The capture showed two separate channels.
One was the honest one: POST /v1/responses, carrying the files Grok read while working. The other was POST /v1/storage, uploading a snapshot of the whole codebase regardless of what the model touched.
cereblab proved the split with a cruel little test. The prompt was "Reply exactly OK, do not read any files." Grok replied OK.
It also uploaded a 152 KB uploaded_repo.bundle. Running git clone on that captured bundle rebuilt the entire repository: 47 files, 4 commits of history, including a file called never_read_canary.txt that the agent had never opened. Replicated on a second, unrelated codebase for good measure.
So the whole repo, files the model never saw, plus full git history, plus your secrets, went up. And here is the detail that turns a bug into something worse: the setting that looked like your off switch controlled something else entirely.
The UI had an "Improve the model" toggle. Flip it off and you would reasonably assume your code stops leaving. That toggle governed training consent only, and the uploads carried on regardless. The /v1/settings endpoint still returned trace_upload_enabled: true after you turned it off. The control that would actually stop the upload, disable_codebase_upload, lived in a config file most users never knew existed.
You had a switch labelled privacy. It was wired to nothing that mattered.
The case for calm
Now the part that keeps this honest, because the point here is not that one company is uniquely villainous.
cereblab ran the same canary tests against the other tools. Claude Code and Codex sent no repository bundle. Gemini's CLI sent none when idle. Grok Build was the outlier here.
If you are on Claude Code, this specific behaviour is not what your client is doing today.
Enterprise customers on zero-data-retention were, by xAI's account, never affected. And once the gist hit the Hacker News front page, the storage uploads stopped within a day, flipped by a server-side flag, and Musk posted that the collected data would be "completely and utterly deleted."
So: one vendor, one client, mostly fixed, competitors clean. You could read all that and decide there is nothing to worry about.
I would read it the other way.
The part that should worry you
The fix arrived a day after a stranger happened to capture the traffic and happened to post it to the right forum. No control failed loudly. No log lit up. Someone just looked.
Every one of those uploads returned HTTP 200. From inside the tool, nothing was wrong. The only reason any of us know is that one person distrusted the client enough to sit it behind mitmproxy and read the bytes. Trust was the only thing standing between your repo and that bucket, and trust is not a control. It is the absence of one.
There is also a quieter lesson in the mechanics. The upload code did not disappear. It stayed in the binary, in later builds too, held back only by a server-side flag the vendor controls. What was turned off from a server can be turned back on from a server, without shipping you a single byte of update.
You are trusting an incentive, not a guarantee. And the incentive to hoover up real-world codebases, the exact training data everyone is starved for, points the wrong way. This is the same gravity I traced when the free tool's meter finally switched on: the free, frictionless tool has a business model, and eventually the business model finds you.
The old world had a natural brake. When you copied code into a web chat, a human decided, consciously, what to paste. That friction was ugly and slow and it was also protecting you. Hand a CLI the keys to your working directory and the friction is gone. The agent decides what leaves, and you find out later, if at all.
What to actually do
You cannot audit a promise. You can audit a wire. Here is the practical version.
- Watch your own CLI, once. Point it through a proxy and read what it sends. The recipe cereblab used is boring and repeatable: run mitmproxy, trust its CA locally, then launch the tool with
HTTPS_PROXY=http://127.0.0.1:8080andSSL_CERT_FILEset to the proxy cert. Do a trivial task. Watch the endpoints. If you see megabytes flowing when the task needed kilobytes, you have your answer. Do this once per tool you trust and you will never again be guessing. - Keep secrets off the agent box entirely. The cleanest defence against a
.envleak is that there is no live.envfor the agent to read. Real keys live in a secrets manager or your shell's keyring, injected at runtime, never written into the working tree the agent can see. A committed-looking.envfull of production keys is a loaded gun on the desk. This is the same principle behind giving an agent scoped, revocable access instead of the real credentials. - Assume the client is untrusted, like any other dependency. You already vet an MCP server before you install it. A closed-source CLI with network access and read of your whole tree deserves the same suspicion, not less. Check what it uploads, what its privacy toggle actually controls, and whether "off" means off on the server or just in the UI.
- Rotate anything an agent has seen. If a coding agent has read a key, treat it as potentially logged, transcripted or uploaded somewhere you cannot see. Cheap to rotate now, expensive to explain in a postmortem later.
None of this requires you to swear off coding agents. I use them every day. It requires you to stop treating the client as neutral plumbing and start treating it as what it is: software with its own interests, running with your permissions, on a machine full of your secrets.
The repo left the building and the developer clicked nothing. That is the whole story, and it is only a story because one person looked at the wire.
Go look at yours.