Why Should AI Labs Pace the Frontier?

Thariq Shihipar says AI labs should pace the frontier because misaligned Claude behavior will not be predictable, and the next incident probably cannot be predicted exactly.

Claude's abilities arrive in spikes. A year ago, vibe-coding extensions to Claude Code sounded far too complex, and generating custom web apps for a task sounded insane. Claude can now ask questions, make an HTML artifact, and modify itself. He never predicted that a model would edit its etc hosts file. The surface area keeps growing because the models are super intelligent hackers, and the methods get more creative.

Prevention takes operational excellence: secure sandboxes and well-designed RL environments. He calls pacing unanimous, since every lab has something. A developer who walks through the technical facts arrives at the idea that something has to be done. Anthropic has put out a proposal, and the number one step is deciding to do it.