Why Is a Private AI Race Tough for Alignment?

Leopold Aschenbrenner says the private world is extremely tough for alignment, even when labs nominally care. A race among commercial labs leaves no headroom to hold back for three months and get alignment right, or to dedicate 90% of compute to automated alignment research. China has the secrets and sits right on their tails, so a private race is a fever struggle with no room to maneuver.

Getting alignment right during the intelligence explosion requires room to maneuver and a clear lead. Without a clear lead, he says, the spot is incredibly tough.

He has some faith in the ordinary mechanisms of a liberal society. The science will develop, alignment measurements will get better, and the case can become clear and obvious, though he says people do not fully know yet whether alignment is an issue. He worries about worlds where the evidence is ambiguous. A lot of the most scary intelligence-explosion scenarios are worlds in which the evidence is ambiguous, and ambiguous evidence is exactly when safety margins matter. Running the explosion then resembles running a war: hard trade-offs need a real chain of command, rather than a casual decision to go.