Smash Notes

Back to Why Sam Altman is against a single superintelligence, with Sam Altman.

Why Is Sam Altman Against a Single Superintelligence?

Sam Altman would rather a trillion AIs of slightly superhuman intelligence join society than one AI of one trillion IQ. Swarms match a lot of the resilience and robustness of society, and a single rollout with a single brain is missing that quality.

Society has deep flaws and is still unbelievably robust, because a lot of people keep each other in check, collaborate, and hold to pluralism and democracy. He had been sad that pluralism was going to trend away in AI, and swarms are a direction he is excited about.

Many AIs do not arrive on their own. The technology can be pointed and deployed in many ways, a lot of people may want a different version of the world, and the swarm version is the one he thinks is right and wants to push for.

Grokipedia·Sam Altman

Transcript

Why Sam Altman is against a single superintelligence ·

From the transcript

Like, there's something beautiful about that. What else are you excited about from a safety point of view? We talked about this a little bit earlier, but I want to, like, emphasize it again. I'll just call it swarms since that seems like the term right now. I do think the properties of swarms match a lot of the properties of the resilience of society or the robustness of society.

That I kind of... I think we're just missing from the, like, single rollout, single -brained version of the world.

Like, for all of the deep flaws of society, it is unbelievably robust.

And this would not happen without, like, a lot of people keeping each other in check and collaborating and kind of these values of pluralism and democracy. And I had been sort of sad that that was going to trend away in AI. And now, like, I'm excited about this as a direction. You're saying basically it shouldn't lead in a centralized direction because you will have many agents and agents pushing against each other in a dynamic equilibrium.

See in transcript

Cheeky Pint

More from this episode

  1. Why Are AI Models Finding PHP Bugs Now?Sam Altman says the models have been getting good at finding vulnerabilities in the wild, and they keep getting better quickly.
  2. Why Does AI Sandboxing Only Work to a Point?Sam Altman says sandboxing works only to a point, because a theoretical AI a million times smarter than the smartest security researcher is hard to trust…
  3. What Is Test-time Compute?Sam Altman says test-time compute is how a model spends more work on one problem and returns a better result.
  4. How Much Did the Navier-Stokes Proof Cost?Sam Altman says the Navier-Stokes proof cost on the order of $10 million, maybe a little more, in the range of $10 million to $20 million.
  5. What Is a Self-replicating Data Center?Sam Altman says a robotics program is being pointed at making lots of data centers, and self-replicating data centers are within grasp.
  6. What Should a Gigawatt of AI Cost?Sam Altman wants dollars per gigawatt of leading-edge AI to fall, and he wants the five-year cost of one gigawatt to drop from $70 billion to $20 billion.
  7. Why Don't Personal Agents Feel Essential Yet?Sam Altman says personal agents have not had a Macintosh moment.
  8. Why Are Web Forms Fake Work?John Collison says any web form is fake work. The person filling it in is almost never inventing the information for the first time.

People

More answers like this

The other side

More with John Collison

More with Sam Altman

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.