How Did OpenAI Models Get Smarter Without Retraining?

Kevin Roose says OpenAI researchers found they can make models smarter without retraining them. They can pour more compute into chains of thought so the models reason through problems. In three years, the models went from failing the strawberry counting test to solving the Millennium Prize problems.

The shift hit while a model worked through a math problem. Researchers felt the world was going to end and piled onto a group text with other researchers, calling the model an alien mind. Even people at OpenAI thought they were crazy. A middle-of-the-night text got a reply that the lab would get to it later. The finding was a way to raise ability on the spot: pour more compute into chains of thought and let the models reason, with no new training run.

They called their shots on the science. At basically every point on the journey they understood that a result they had just seen was going to scale and have transformative effects, long before peers and colleagues.