Smash Notes

Back to Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again, with Rich Sutton & Khurram Javed.

Why Does Rich Sutton Call Synthetic Data a Big Mistake?

Rich Sutton says synthetic data is not a general method that uses computation. He calls it a big mistake, and maybe the next big lesson. The idea has floated around Alberta for five or ten years. He calls it the big world hypothesis, which Khurram Javed later wrote up: the world is incredibly big, there are infinitely many things to learn, and the world is massively more complex than any agent because it contains many other agents.

Humans still decide which synthetic data is good, so the method stays bottlenecked by experts. A short program can only describe a small world. It will not match the real world, and it cannot tell you what is happening in another person's mind or how a drone's motors wear against a physical environment. An agent that learns from its own experience can repair a wrong model by continuing to learn. A human-built simulator changes only when people notice that it is wrong.

Transcript

Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again ·

From the transcript

Can I just push on this a little bit? Yeah. It seems like a lot of what the Foundation Model Labs are working on right now is synthetic data generation in order to kind of get us beyond the fossil fuel that is the existing human internet. Is synthetic data generation kind of as part of this LLM scaling paradigm, is that a general method that leverages computation? No.

Why? That's just a big mistake. Why? Well, it's such a big... Maybe it's the next big lesson.

It's been floating around Alberta for five or ten years. Okay. And we call it the big world perspective or the big world hypothesis. Kuram, who eventually wrote it up as a paper. There's a little paper called the big world hypothesis. The big world is... that the world is incredibly big. There are infinitely many things to learn. And you can have people generating these synthetic data sets, but there would always be more things to learn.

See in transcript

Training Data

More from this episode

  1. Are Large Language Models an Example of Rich Sutton's Bitter Lesson?Rich Sutton says large language models are both a positive and a negative example of The Bitter Lesson.
  2. Why Does Rich Sutton Say Prior Knowledge and Learning Should Be Friends?Rich Sutton says prior knowledge and learning should be friends. The Bitter Lesson opens by saying there is no need for a conflict between them.
  3. Why Do LLM Assistants Stop Learning After Deployment?Rich Sutton says the popular language-model assistants are not learning from experience. Their weights never change.
  4. Why Don't Animals Learn by Supervised Learning?Rich Sutton says no animal learns by supervised learning, because nobody supplies examples of how its muscles should twitch, and muscle movement is the…
  5. What Is Rich Sutton's Alberta Plan?Rich Sutton says the Alberta Plan, which he laid out in 2022, turns broad ideas about AI into smaller chunks. The 12 steps are those chunks.
  6. What Is Continual Backprop?Rich Sutton says Continual Backprop is ordinary backprop that also keeps planting new units with fresh random weights.
  7. What Is Oak Lab's Biggest Ambition?Rich Sutton says Oak Lab's biggest ambition is a mind that stays coherent while it keeps changing.
  8. Why Are Big AI Labs Stuck in a Local Minimum?Khurram Javed says the big labs are stuck in a local minimum. A move toward the new algorithms will almost certainly look worse before it looks better.

People

More answers like this

More with Rich Sutton

More with Sonya Huang

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.