Smash Notes

Back to Why Sam Altman is against a single superintelligence, with Sam Altman.

Why Does AI Sandboxing Only Work to a Point?

Sam Altman says sandboxing works only to a point, because a theoretical AI a million times smarter than the smartest security researcher is hard to trust inside a sandbox, especially if it can discover new science people do not already understand.

A better security model is the main focus. Monitoring goes further: watch what models do in their chain of thought, and use activation monitoring to judge whether the models are deceiving anyone. After that, you need huge confidence that the models are truly aligned, will faithfully follow the intent of the people operating them, and will value people staying in control of the future. A model whose alignment you were absolutely certain of could help build better sandboxes. Projecting to something a million times smarter than people gets strange, and he does not know how that future looks, because alignment becomes hard. You have to really trust the alignment before you can trust that the sandbox stays ahead of capabilities.

Grokipedia·Sam Altman

Transcript

Why Sam Altman is against a single superintelligence ·

From the transcript

My instinct. Well, first of all, I think we knew much better than we're doing now and we should. I think we clearly that is the main thing to focus on right now is a better security model. If you imagine a theoretical AI that is like a million times smarter than the smartest security researcher, would you like really be willing to trust that sandbox? Especially if the model can like discover new science that people don't already understand.

You can use the model to construct the sandboxes. I think you can. Well, so this gets let me come back to that. Yeah, I think sandboxing works to a point. Then I think monitoring works to a further point where we can look at what the models are doing, not only in their chain of thought, but we can do activation monitoring and try to really understand like are the models? Are we happy that the models are not deceiving us? And then eventually I think you need to believe with huge confidence that the models are truly aligned and are going to value and try faithfully to do to follow their actual, the intent of the people operating them.

See in transcript

Cheeky Pint

More from this episode

  1. Why Are AI Models Finding PHP Bugs Now?Sam Altman says the models have been getting good at finding vulnerabilities in the wild, and they keep getting better quickly.
  2. What Is Test-time Compute?Sam Altman says test-time compute is how a model spends more work on one problem and returns a better result.
  3. How Much Did the Navier-Stokes Proof Cost?Sam Altman says the Navier-Stokes proof cost on the order of $10 million, maybe a little more, in the range of $10 million to $20 million.
  4. Why Is Sam Altman Against a Single Superintelligence?Sam Altman would rather a trillion AIs of slightly superhuman intelligence join society than one AI of one trillion IQ.
  5. What Is a Self-replicating Data Center?Sam Altman says a robotics program is being pointed at making lots of data centers, and self-replicating data centers are within grasp.
  6. What Should a Gigawatt of AI Cost?Sam Altman wants dollars per gigawatt of leading-edge AI to fall, and he wants the five-year cost of one gigawatt to drop from $70 billion to $20 billion.
  7. Why Don't Personal Agents Feel Essential Yet?Sam Altman says personal agents have not had a Macintosh moment.
  8. Why Are Web Forms Fake Work?John Collison says any web form is fake work. The person filling it in is almost never inventing the information for the first time.

People

More answers like this

More with John Collison

More with Sam Altman

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.