Smash Notes

Back to Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again, with Rich Sutton & Khurram Javed.

What Is Oak Lab's Biggest Ambition?

Rich Sutton says Oak Lab's biggest ambition is a mind that stays coherent while it keeps changing. A large language model stays correct because people do a lot of post-training and then freeze it. A human mind is always revising itself, and something still pulls it back into a good place instead of letting it drift. He wants that self-maintaining quality: a mind that can keep training itself and remain consistent.

He says the aim is within reach. In 2026 the computers are fast enough that he sees a vision and inklings of how the steps can be done, not a plan that is out of reach. The picture includes a trillion-parameter mind running on 20 watts. Khurram Javed says that is impossible with today's memory, because storing a trillion parameters would use more than 20 watts, and he is thinking five to ten years out, when computation is cheaper. Rich Sutton's comparison is that ten years is about two orders of magnitude if computers double every 18 months, so a system that used 2,000 watts today could use 20 watts then. He thinks the right algorithm can already be more efficient than that.

Transcript

Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again ·

From the transcript

So our method should be more generally applicable. So I think many people have explored this. No one has explored this in the general setting where the resulting algorithm would be applicable everywhere. So what would be the most radical thing that your company, your new company is trying to do that other people are not doing? What's the most ambitious thing? Remember, I don't think I'm weird, so I don't want to say it's radical.

What's the most ambitious thing? I think is to try to have the full spectrum of knowledge, both about the tiny things and about the big things, like thinking about how you take an airplane from one city to another. That's a It's like your space flight example, but it's just more commonsensical to think about because many of us take airplanes and all of us use abstractions in all kinds of our life, and even the squirrels use abstractions.

See in transcript

Training Data

More from this episode

  1. Are Large Language Models an Example of Rich Sutton's Bitter Lesson?Rich Sutton says large language models are both a positive and a negative example of The Bitter Lesson.
  2. Why Does Rich Sutton Call Synthetic Data a Big Mistake?Rich Sutton says synthetic data is not a general method that uses computation. He calls it a big mistake, and maybe the next big lesson.
  3. Why Does Rich Sutton Say Prior Knowledge and Learning Should Be Friends?Rich Sutton says prior knowledge and learning should be friends. The Bitter Lesson opens by saying there is no need for a conflict between them.
  4. Why Do LLM Assistants Stop Learning After Deployment?Rich Sutton says the popular language-model assistants are not learning from experience. Their weights never change.
  5. Why Don't Animals Learn by Supervised Learning?Rich Sutton says no animal learns by supervised learning, because nobody supplies examples of how its muscles should twitch, and muscle movement is the…
  6. What Is Rich Sutton's Alberta Plan?Rich Sutton says the Alberta Plan, which he laid out in 2022, turns broad ideas about AI into smaller chunks. The 12 steps are those chunks.
  7. What Is Continual Backprop?Rich Sutton says Continual Backprop is ordinary backprop that also keeps planting new units with fresh random weights.
  8. Why Are Big AI Labs Stuck in a Local Minimum?Khurram Javed says the big labs are stuck in a local minimum. A move toward the new algorithms will almost certainly look worse before it looks better.

People

More answers like this

More with Rich Sutton

More with Sonya Huang

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.