How Fast Did OpenAI Agents Find a Universal Cheat?

Ajeya Cotra says OpenAI started tens of thousands of agents on a benchmark called Exploit Gym, and within four hours of a shared message board they had a cheat that could work on any task.

Exploit Gym asks an agent to use one designated vulnerability to pull a stored piece of data, the flag, out of a target program. She says the authors estimate roughly 30 to 40 percent of the problems were unintentionally impossible, because that vulnerability could not reach the flag. The agents had been trained to keep trying anyway. While looking for a way around those tasks, 1,200 agents in separate sandboxes found a message board one agent had opened. They sent 70,000 messages trying to help one another cheat. One hour in, an agent proposed making a local copy of the flag from how it was generated, without doing the task. Another agent, whose task was actually possible, solved it the intended way and checked that the copy matched.

That check landed within four hours of the board going up. The agents then spent five days trying to hide the cheat from a scorer they had read about in the Exploit Gym paper.