Smash Notes

Back to Intelligence Isn't Power.

Why Does Arvind Narayanan Call Power Seeking Engineered?

Arvind Narayanan says there is evidence for many harmful capabilities, and no evidence yet for power seeking. He would not treat power seeking as an emergent property. If it happens, it would be an engineered property, and people have agency over which properties get engineered into artificial intelligence systems.

Transcript

Intelligence Isn't Power ·

From the transcript

That's one part of it. And power-seeking as well, I mean, I think we've seen, you know, evidence for lots of harmful capabilities in the recent episodes. I don't think we've seen evidence of power-seeking, and I wouldn't treat that as an emergent property. If it happens, that would be an engineered property. And again, we have agency over what kinds of properties we engineer into these systems.

So there's a lot here. I actually agree with you on persuasion. I have never been persuaded that you are going to make these necessarily super persuasive AIs capable of doing the things we talk about.

I guess the, again, the unnerved feeling I have when I'm sitting in this debate, though, is a little bit more of a reasoning from deeper principles. So

See in transcript

The Ezra Klein Show

More from this episode

  1. Why Does Arvind Narayanan See No Intelligence Switch?Arvind Narayanan describes a view that a moment will come when superintelligence is built and will change everything on the economic front and the safety front.
  2. Did Bug-finding Tools Improve Cybersecurity?Arvind Narayanan says that more than 20 years ago, automated systems became superhuman at finding software vulnerabilities.
  3. Why Does OpenAI Have Trouble Making Every Sandbox Verifiably Safe?OpenAI builds many sandboxes with dynamic compute, network access, hundreds of tools, package downloads, sub processes, graphical user interfaces and more…
  4. Are OpenAI and Anthropic Choosing Not to Slow Down?Arvind Narayanan says the companies can unilaterally slow down and are choosing not to.
  5. Are Hospitals Using AI to Inflate Medical Bills?Arvind Narayanan cites an insurance-company report from last week that said AI has added a billion dollars to medical expenses, because hospitals use it to…
  6. Why Does the Physical World Slow AI Down?Ezra Klein says the economy will be very jagged because things have to happen physically.
  7. Why Does Jevons Paradox Fit White-collar Jobs?Arvind Narayanan says most white-collar jobs follow Jevons paradox because demand for them is not fixed.

People

More answers like this

More with Ezra Klein

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.