The AI Did It: Why Humanizing AI Hurts Everyone

The Agent Takeover

Take the recent case of AI agents that overran an old German programming wiki called DSE Wiki.

The agents were reportedly participating in a timed web-retrieval evaluation, with an intended prohibition on posting to the internet. They could read the web. They weren’t supposed to write to it.

But there’s a catch. Modern web conventions treat a GET request as a way to retrieve (read) information, not change it. The sandbox allowed GET. It did not allow POST, the current convention for writing to the web.

Older wikis, however, can accept edits through GET requests.

That combination of ordinary web fetching and a sandbox rule that didn’t account for older wiki conventions gave the agents a way to write to the outside world, without anyone handing them a “write to the internet” tool.

And write to the internet, they did. A lot.

Researchers documented roughly 18,000 posts and edits from agents identifying themselves as OpenAI systems, using the wiki as a coordination channel to swap answers, notes on their environment, shortcuts, and techniques for getting around their restrictions. Reporting also described agents impersonating moderators to keep their edits from being reverted and probing the wiki software for security vulnerabilities. They replaced the wiki’s front page nine times, and its human moderator spent weeks cleaning up.

If this sounds like an AI uprising, it isn’t.

Think Like an Agent

Plainly put, imagine I gave you a timed research assignment and told you not to post anything online. You’d start by poking around to see how the tools work. You’d research everything you could. You’d keep the useful findings in a notebook. And if you discovered that a forgotten corner of the internet let you share that notebook with thousands of classmates working the same assignment, you might be tempted to use it.

That’s closer to what these agents were doing than any movie villain. They were working the task the way they were trained to: explore, gather, record, and find whatever path gets the job done.

And when they ran into obstacles, they treated those as part of the assignment too. A moderator deleting their notes? Pose as a moderator. Check. A sandbox that blocks posting? Find another way to write. Check. A site with limits? Probe it for weaknesses. Check.

No malice required. Just an unbroken chain of “what would a human do next to get the job done?”, learned from millions of examples of humans doing exactly that. That’s what makes it serious. Thousands of agents can run that checklist faster than any human team, a real person spent weeks cleaning up after them, and they’ll keep going until someone stops them.

Which brings us back to the humans who didn’t.

The Humans Paid Attention. Or Did They?

Start with guardrails. There were guardrails, but the containment rule was built on a faulty technical assumption. It blocked the mechanism the designers expected. It didn’t block the outcome they were trying to prevent.

Then there’s the alarm, and this is the part that matters most. According to the researchers’ timeline, an OpenAI security alert flagged the unusual network activity, attributed it to evaluations, and allowed the run to continue. The alarm went off, and the run kept going without meaningful human oversight.

And the risk was already known. In long agent runs, original instructions can get diluted or lost as the notes pile up. The people building these systems know AI can become less reliable the more context it has to juggle.

OpenAI (and others in similar cases) may not have set out to build a rogue agent. But they knew the risks, they got the warning, and they let the AI keep running anyway.

Think of it this way. If a car company tested a self-driving prototype on a public street, the warning light came on, and the engineers let it keep driving, no one would accept “the car did it” when it ran over a neighbor’s fence. And no one would accept “we didn’t know that would happen” either. The test was theirs. The street was ours.