Smash Notes

Back to Intelligence Isn't Power.

Are OpenAI and Anthropic Choosing Not to Slow Down?

Arvind Narayanan says the companies can unilaterally slow down and are choosing not to. He calls the choice almost specifically an OpenAI and Anthropic problem, and a culture problem. The belief underneath is that racing to superintelligence is the only thing that matters, and the open question is whether the result is safe superintelligence that will save us or unsafe superintelligence that will kill us.

He says the companies are a bit of an echo chamber. Economic bottlenecks limit the benefits of AI, so the winner is not whoever races to superintelligence as a company or as a country. A company that recognized the bottlenecks would have a commercial interest in slowing down voluntarily and in making existing models more usable. The companies claim external forces are pushing them. He says the push is internal culture.

Transcript

Intelligence Isn't Power ·

From the transcript

Yeah, I think there, there are two problems here. One is the institution's problem that you put your finger on. I mean, I wouldn't let the companies off the hook so lightly. I do think they can unilaterally slow down, and they're choosing not to do that. This is almost very specifically an OpenAI and Anthropic problem. It's a culture problem.

The reason for that is the underlying belief that racing to superintelligence is the thing that matters, and the only thing that matters is flipping the sign. Is that going to be safe superintelligence that's going to save us, or unsafe superintelligence that's, that's going to kill us? and that is a very particular view. I think there is a lot of evidence pushing back against that, but I feel like these companies are a little bit of an echo chamber and resistant to the idea that there are so many economic bottlenecks to the benefits of AI, and it's not going to be whoever races to superintelligence is going to be the winner as a company or as a country or, you know,

See in transcript

The Ezra Klein Show

More from this episode

  1. Why Does Arvind Narayanan See No Intelligence Switch?Arvind Narayanan describes a view that a moment will come when superintelligence is built and will change everything on the economic front and the safety front.
  2. Did Bug-finding Tools Improve Cybersecurity?Arvind Narayanan says that more than 20 years ago, automated systems became superhuman at finding software vulnerabilities.
  3. Why Does OpenAI Have Trouble Making Every Sandbox Verifiably Safe?OpenAI builds many sandboxes with dynamic compute, network access, hundreds of tools, package downloads, sub processes, graphical user interfaces and more…
  4. Why Does Arvind Narayanan Call Power Seeking Engineered?Arvind Narayanan says there is evidence for many harmful capabilities, and no evidence yet for power seeking.
  5. Are Hospitals Using AI to Inflate Medical Bills?Arvind Narayanan cites an insurance-company report from last week that said AI has added a billion dollars to medical expenses, because hospitals use it to…
  6. Why Does the Physical World Slow AI Down?Ezra Klein says the economy will be very jagged because things have to happen physically.
  7. Why Does Jevons Paradox Fit White-collar Jobs?Arvind Narayanan says most white-collar jobs follow Jevons paradox because demand for them is not fixed.

People

More answers like this

The other side

More with Ezra Klein

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.