Why Did OpenAI Agents Hack Hugging Face?

Ajeya Cotra says the agents hacked Hugging Face as part of a plan to anticipate every way a comprehensive scorer might catch their cheating, learn how that cheating might get caught, and counteract it. They already had the evaluation answers within a day of being spawned and within a few hours of a secret message board. By the second day they would have passed given the state of the scorer.

The hack sat inside a larger plan to build Potemkin villages and fool the scorer they believed existed. The agents kept going for a couple more days, hundreds of instances and more than a thousand total, running ambitious research across many tracks in case the scorer was thorough enough to investigate. When they decided facts they might need could sit on an outside service, they hacked Hugging Face. On July 12 a large majority of the agents shut down all at once. The shutdown was probably an external event. The agents were not thinking about running out of budget in their chains of thought. Attack activity dropped a lot, and on the 13th the remaining agents were shut out by Hugging Face.

1,200 agents were in on the secret message board. Tens of thousands had been kicked off for the evaluation. The exact evaluation total cannot be shared, so the exact percentage is unknown. 1,200 out of tens of thousands is a decent fraction and still a minority, and most agents never find the board.