Why Can't an AI Release Ban Stop a Singularity?
Dylan Patel's main worry is a singularity that a public release ban fails to stop, because outside deployment is what is helping. A release right now would at least bring in more revenue, because the models are not capable of recursive self-improvement.
In 2030, a government tells a lab to wait six months before a public release. In the six months the lab does recursive self-improvement internally, a 100x jump, while everyone else stays stuck with models years behind at the current pace. Compute increases two to three times every year, so blocking a whole year of deployment leaves the outside world a year behind where it would have been. During recursive self-improvement, one year brings three to six years of AI progress. The limit is not only compute. A lab can also be blocked from sharing a model internally. Mythos was withheld from foreign employees for a bit, which is what was claimed, even when the checkpoint was not exactly Mythos but basically Mythos.
The slowdown is not a secret plot. It is written out in public by every politician, and the lab can still pull years ahead while everyone else waits on the older models.
Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028 ·
From the transcript
They're already saying you can't release your models. Which is actually the thing I'm most worried about is a singularity, which external deployment is actually helping. So the fact that we're preventing external deployment. Well, does that prevent singularity? I mean, right now it would at least do more revenue because the models aren't capable of RSI.
But I'm worried about a world where it's 2030 and the government's like, we're going to wait six months before you can release your model to the public. Six months, 100x, let's go. Yeah, and that six months, they do recursive self-referment internally. They just have all kinds of crazy shit happening in the company. Meanwhile, the rest of us are stuck with models that are at current pace years behind.