Why Isn't AI Automating the Easy Work Yet?
Diogo Almeida says models have been smart enough for easy automation for years, and that intelligence still is not being put to work. A document he wrote at OpenAI was titled "we could have had AGI a year ago," after OpenAI's definition of doing most of the world's economically valuable work. He reads the spirit of that line as work by volume. Models from the GPT-3.5 era, and possibly the GPT-3 era, could have met it. They were already past a median level on all sorts of tasks.
That should have meant tons of automation, at least on the easy work. GPQA, the hard multiple-choice test he describes as PhD-level, was passing that bar in 2023. He calls it tragic that intelligence at that level was not put to work, and says the raw material has been around for a long time.
Ep 97: Jev Creator on Decision Models, Benchmark Fatigue & Why the Labs Missed It ·
From the transcript
it would have been harder, but doable. Back when I was more naive, oh man, I sound like an old dude. When I was more naive and I thought that this would take a week of work, What convinced me, I love writing documents. I think it's a great way to think. I wrote a document at OpenAI that was titled, we could have had AGI a year ago, at least by OpenAI's definition of AGI, which I don't think that people really listen to the letter of the definition, but I think it's actually a really great letter.
I think that models starting from GPT-3.5 era, possibly even GPT-3 era, could have met the definition of AGI at the time, which is still the same, like doing most of the world's economically valuable work by volume. It doesn't say by volume, but I assume the spirit of that is by volume. Because the models are so unbelievably smart. Like GPT-3.5 level models surpassed median level intelligence at all sorts of things.