How Fast Does Recursive Self-improvement Speed Up AI?

Noam Brown expects recursive self-improvement to speed AI progress a lot, but not 100 times faster overnight. Serial experiments and a limited supply of graphics processing units still cap how fast research can move, even with brilliant models running the work.

A team of the most brilliant people at OpenAI with 100 times less compute would make less progress than the current team with current compute, though not 100 times less. The same kind of limit shows up once models run experiments on the compute already in hand. New models take a while to train, and results take a while to come back, so the experiments run one after another. Graphics processing units have to be available to run them. Progress would still go a lot faster. On the current exponential, a pace 3 times faster is massive, and 100 times faster is a different scale.

He has some confidence and is not 100 percent confident. An overnight intelligence explosion is possible. The speedup could be only 50 percent. There is a lot of uncertainty.