Demis Hassabis says society is “not ready” for the AI future

0:01

We've all spent money on something that seemed incredible but didn't live up to the hype. A kitchen gadget, a car. When it doesn't deliver, the disappointment is real. Marketers suffer from this too, chasing metrics that look great but don't translate into revenue. LinkedIn calls that bull spend. Invest in what matters and reach the right buyers with LinkedIn ads.

0:24

You can target by company, industry, job title, and more. LinkedIn ads also generates the highest ROAS of all major ad networks. Spend $250 on your first campaign on LinkedIn ads and get a $250 credit for the next one. Just go to linkedin.com slash scale. That's linkedin.com slash scale. Terms and conditions apply.

0:49

AI is full of possibilities. Enterprises run on realities. That's why Pega combines the world's most powerful AI with the governance, transparency, and control enterprises demand. So you can move faster, adapt faster, and scale AI with confidence. All the power of AI, none of the uncertainty. Learn more at pega.com slash AI.

1:19

A lot of the cyber issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. The human's very much still in the loop. And the next phase, which I think carries the most risk, is closed RSI, closed loop, without the human reviewer process. And that will obviously be faster, but I think is also a very dangerous way to get to full AGI.

1:44

I do think there are some things that generally all the leading labs do agree on, like the idea of third party auditors or evaluators coming in. Obviously, there's a question about which third parties. And I think we need a healthy kind of ecosystem of that. You could have good startups and companies doing that. There's enough revenue there for them, given the amount of investment going into AI as a whole.

2:06

This is Masters of Scale. I'm Jeff Berman, your host. In today's episode, our very own Reid Hoffman sits down with the incredible Demis Esavis. Demis is the co-founder and chair of Google, DeepMind, and chief scientist of Alphabet. Reid and Demis discuss the race to AGI, the future of AI-fueled scientific discovery, the need for more cooperation on safety efforts, and much, much more.

2:31

I hope you enjoy it. Demis, thank you for joining us. Great to be here. Thanks for having me. We've known each other for, I think, well over 10 years now, maybe approaching 12 or something like that. It was an honor when you asked me to be on the AI ethics board for Google for our various funny meetings. Yes. So let's start, because one of the things we've been trying to do at the summit is to be envisioning the world as we'd want to make it.

3:14

The theme has been agency. And, you know, as you have been a leader in and knew well before most people were thinking about AI, that one of the key things for thinking about AI for human benefit is medicine and medical. And so say a little bit about how you envision this, how this helps people. People say, oh, if I could be getting that, I understand why you AI guys are actually trying to build something that creates a longer human experience, a more quality of experience.

3:45

Call it anti-X-risk. Yeah, for sure. I mean, if we take a step back for a moment, it's like big picture. And you know this, knowing us from quite early in our DeepMind journey. The whole point of me working on AI for my whole career was to use it as a kind of like the ultimate tool to advance science and medicine and our understanding of the world around us, but also improving human health.

4:05

I've always said that the number one thing we should use these amazing tools that we've now built and we were, you know, dreaming about 20 years ago is to improve the human condition and improve human health. And I've tried to do that certainly with my own more personal projects, obviously things like AlphaFold, but also the spin-out drug discovery company Isomorphic.

4:26

I've tried to push it really hard in that direction, as I know you have too more recently. And I think it's exactly the sorts of things we need to be showing the world that AI can do. I think what better thing than to use AI to help us cure terrible diseases. So some of the protein folding was this kind of intersection of the idea of like, we have really good data, we could use scale learning, because the whole learning, the cognitive revolution from learning, starting with even Pong, but the, you know, and self play and all the rest is like, okay, that's, this would be a really good way of doing it.

5:00

How does AlphaFold lead to medicines? Because one of the things is a question I get frequently is like, yeah, you're talking about that, but is it true? Is it going to happen anytime soon? You know, so I'm passing along the question I get to you. Yes, sure. Well, look, biology is the most complex systems in the universe, right? So that's why it's so hard, because of the emergent behavior, the interactions.

5:23

And proteins are the building blocks of life. That's how I've always seen them. And the protein folding problem, which is what we tried to solve with AlphaFold, was considered to be like a 50-year grand challenge in biology. It's sometimes described to me as like Fermat's last theorem, but for biology. And it's critical to understand the structure of a protein from its genetic sequence, its amino acid sequence, to understand its function.

5:48

And that's really important to understand basic biological mechanisms, but also to understand what goes wrong in disease and what things to target. You know, if you imagine you're designing a drug, then you need to know what the 3D surface of the protein looks like so you know which part of that protein to bind to. And so it was always a critical...

6:08

but not obviously, only still a small part of the entire drug discovery process. But it's an absolutely critical part of that. And we were able to, you know, with AlphaFold, accelerate it by orders of magnitude by building AlphaFold. And interestingly, at the time, and if you think about it, we finished AlphaFold 2 back in 2020, which is, you know, six plus years ago now.

6:30

And it's still probably the best example of concrete example of of AI applied to a really hard scientific problem that was thought to be sort of uncrackable. And so that just shows you how far ahead of its time it was. And now, you know, we're really proud of the impact it's made. Like four million researchers around the world use it, pretty much every biology and medical researcher in the world, almost every drug.

6:51

in future will have made use of AlphaFold, all the structures, but it's not enough. We want to expand it to biochemistry and also predicting things like toxicity or acne properties, all of these other things that you need to know about before you can have a drug that's safe to use and that's efficient in whatever it is you're trying to tackle.

7:10

So speaking of accelerations, let's go to one of the hot topics of the day, recursive self-improvement. What is your kind of theory of the game? And I say this because you are a master games person, so I use that as kind of deliberate. And if you don't know what I'm referring to, either look at the documentary or look it up on Wikipedia. So like RSI, opportunities, concerns.

7:35

Yeah. Well, look, RSI, it's obviously an extremely useful technique to make progress. It mostly works best in what I call verifiable domains, what's called verifiable domains. So math is one, which is why we're seeing so much advance there. Coding's another. But originally, it was games. So actually, one of the first sort of famous recursive self-improving systems was one we invented quite a lot of years ago now called AlphaZero, built off the back of AlphaGo, which was the kind of famous program that beat the world champion at Go back in 10 years ago now, 2016, and kind of was the beginning, I think, of the modern AI era.

8:16

And then we built a second version that was in some ways more consequential, AlphaZero, that didn't need to use any human data anymore. And it just basically played against itself, started at random. And then after around a dozen to two dozen different epochs, it played like a million games against itself. It would then train a new version of itself that was slightly better than the old version.

8:41

And then that would play a million games against itself and so on. And for example, and then it could play any game, not just Go. It could play any two-player perfect information game. And for me, it was especially visceral as a chess player because I remember watching it play chess. And within, you know, 12 hours, basically, or less than 12 hours, it would start from random play in the morning.

9:03

to you know you'd have your coffee break at 11 a.m and i could still just about beat it then by lunchtime it was grand master level and then by by tea time it was better than any chess playing entity dinner time for the us yeah computer time or computer but you know kind of handcrafted chess computer is better than all of them so we sort of saw the power of that recursive self-improvement and obviously the thing that all these domains have in common is you can verify the answer Is it a correct answer? Is it an optimal answer? Is the equation correct? Does the code run? Did you win the game? So those domains are...

9:38

And the important thing about a verifiable domain is you can verify the answer, so you know when you've hit the right point. But also, that means you can create almost infinite synthetic data. because you can just use the current best versions you have to set new problems or make new moves and then just sort of verify whether the synthetic data was correct.

10:00

The problem with synthetic data a lot of the time is you can't verify that it's right from the right distribution. But in these cases, you can. So we've seen the enormous progress with maths and with coding, and coding's obviously become enormously valuable, and everyone is having to do that now. But I do worry about it because in the limit, so there's different forms of RSI.

10:22

Right now we're in the early days of RSI, I would say, where what's happening is, and you're probably all experiencing this as well with your teams, is it's making a human engineer maybe three, four, five times more productive. But the human's very much still in the loop and reviewing code and architecting things and writing code still, but it's all just enhanced.

10:47

The next phase, which I think carries the most risk, is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process in the whole procedure. And that will obviously be faster, but I think is also a very dangerous way to get to full AGI because we're likely for two reasons. It's sometimes used to be referred to 20 years ago as like an intelligence explosion because you don't quite know, you know, how do you control, keep control of such a system, such a process.

11:18

And then the end result will likely be quite alien. It's going to be like an alien intelligence. There's no reason you to think it would be sort of in the similar kind of mind space that humans are in. So that will make it very hard for us to understand it and maybe for it to understand us. So I think we should be very cautious as we get closer towards that closed loop form of RSI.

11:41

And I think all the AI researchers sort of know that, but there is a sort of race to the, there's this sort of intense, obviously competitive race to improve your coding systems. So presumably the, the incentives to be in a race continue. And you and I, some people really know this, have tried to figure out if there's various ways you can get global collaboration to have less.

12:05

You've dedicated a huge amount of energy to that, so thank you. It's a hard game, as you know. But presume that the competitive pressure for continuing to do RSI and coding continues. In that environment, what are the best ways that we can potentially navigate that? Because we have to see kind of where RSI is going, what it's pointed at. What are the kinds of things that the leading labs, such as your own and the others, what should they be doing to try to navigate this environment? Well, what we're doing and I think other leading labs are trying to do is you've got to– so there's more prosaic things like make sure cyber defense is accelerating faster than cyber attack

12:47

and making sure you give access to trusted cyber defenders first on maybe slightly fine-tuned versions of your general models that are–

12:57

fine-tuned for cyber defense. And so you keep the defenders ahead of the attackers. It's a pretty tricky problem to do in general. Also, building much more hardened sandboxes. I think a lot of the cyber kind of issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. I think better, you need really amazing monitoring systems.

13:18

You know, this shouldn't be possible for agents to be running for weeks without anyone realising. You know, these kinds of things that we've seen some of our competitors and others, maybe, you know, it's just the beginning of this happening, so I think everyone will get rapidly better, but they have to as well, because otherwise something bad's going to happen inadvertently, some kind of incident, you know, I sometimes call it like an oil spill, is going to happen.

13:40

On the current trajectory, if we're not, if we don't get collectively, if if everyone doesn't pay a lot more attention and put a lot more resources on these types of systems. Eventually, you kind of want formal guarantees, maybe some bounds around what these systems can do and how robust, test how robust the things like the guardrails are.

13:59

This is actually something I encourage a lot of top mathematicians that I know well who are maybe pretty sad about the state of pure maths, But their skills would be incredible if they pivoted to these types of domains, more like complexity theory, game theory. And of course, we're going to have to start dealing with swarms of these agents, potentially collaborating in some way.

14:21

How does one keep track of all of that? Should there be ways that the labs are encouraged, required, permitted to collaborate on this? And if so, what should some of those ways be? Because it obviously gets stronger if there is sharing of intelligence, methodology, you know, mutual accountability. Yeah, I mean, there's been lots of proposals and I've been working on these things a lot.

14:42

And, you know, you saw my essay over the summer. Maybe some of you read it about potential for like an SRO industry run standards body with some kind of government oversight over it. I think something like that is going to be needed. And then ideally it becomes international. You know, it's US led. And then obviously the state of geopolitics today makes it and our international institutions are very, very weak, as we all know.

15:06

just at the time when we need them to be strongest, in a way. So that's a bit unfortunate, but we've got to start somewhere. By the way, British, a bit unfortunate is, holy f***, just to translate. Just to translate, another translation. Yes, like tea time and dinner time. Yes, exactly, a bit unfortunate.

15:25

And so we're trying to, you know, various of us are trying to do things about that. And that essay did have an impact, you know, in the White House and in the administration and with our colleagues. And eventually, you know, I think some of that energy became the accord that the famous letter everyone signed, which is... Which is better than nothing.

15:43

I don't think it goes far enough, but it's a start. And at least it acknowledges from all of the main players the issues that are there and that they need to be addressed. It doesn't say too much about what to do about them, but that has to come next. And at least it gives air cover, I think, for the teams to work together on what sorts of benchmarks, what are best practices on safety, what are you seeing in terms of analysis of what things could happen with these types of systems.

16:08

And I do think there are some things that generally all the leading labs do agree on, like the idea of third party auditors or evaluators coming in, obviously there's a question about which third parties. And I think we need a healthy kind of ecosystem of that, that actually, you know, you could have good startups and companies doing that, that there's enough revenue there for them.

16:27

And there should be given the amount of investment going into AI as a whole, a bit like there are auditors, you know, in the finance industry or in any of these industries. So I could see that happening. And some of the big players are moving into this, like Accenture. We're working and other leaders are working with, I think they're a little bit ahead of the game because they've acquired, you know, a good startup, good friend of ours, Mark Warner Startup.

16:45

So they have some expertise internally. But I think we need more of that. And then other things like, you know, the leading labs may be red teaming each other before general release of these systems.

17:03

We all know the disappointment that comes when a grand gesture falls flat, something that seemed impressive in the moment but didn't deliver the results you expected. Marketing budgets too often fall into that trap, aiming for big impressions, big reach, but without the revenue to back it. LinkedIn has a name for it, Bullspend. LinkedIn ads generate the highest ROAS of all major ad networks.

17:26

They target by company, industry, job title, and more. Spend $250 on your first campaign on LinkedIn ads and get a $250 credit for the next one. Just go to linkedin.com slash scale. That's linkedin.com slash scale. Terms and conditions apply.

17:46

Every business hits the same wall the moment your business outgrows the systems that got you here. Spreadsheets everywhere, five different logins, data scattered across entities that don't even talk to each other. If your finance team spends more time finding data than using it, that's when you need the Intuit ERP. Intuit Enterprise Suite is the AI-native ERP solution that's powerful, painless, and proven, built for businesses scaling fast without the complexity that usually comes with it.

18:18

From the makers of QuickBooks, learn more at intuit.com slash ERP.

18:26

Hey, listeners, Bob here. If you listen to Rapid Response on Masters of Scale, you may be missing half the show because every Friday we release a second Rapid Response exclusively in the Rapid Response feed. The guests and topics are just as compelling and timely from Ford CEO to NASA's administrator to the lessons from the Devil Wears Prada.

18:50

It takes about 10 seconds to find. Just search Rapid Response wherever you listen to podcasts and hit follow to make sure you never miss an episode. I hope to see you there.

19:06

So let's move from that to a little bit more of the fantastical. So, you know, it started actually a few years back. I think it was a Google engineer who said, I asked Gemini if it was conscious and it said, yes. So therefore it's conscious, which seems like a little bit of

19:20

a weak inference. But that's now gaining some motion with some of the things especially Anthropic is doing and the way that it kind of strategizes thinking that at some point, it's almost like a Pascal's wager, at some point there will be consciousness in these systems and we want those conscious systems to look back saying that we were treating them the same way that we treat a two-year-old, which is, hey, we were good to you as a two-year-old, and so it kind of continues through.

19:47

Obviously, to many people, that looks kind of wacky, right? And so what do you think, I mean, because part of what I think maybe many people don't realize is you also come to this from a depth of expertise in brain science. And, you know, one of the things is not just a, oh, did the chat bot say it was conscious or not is not a way to make a decision on this kind of thing.

20:07

What do you think some of the ways to think about this is and also how we should be deliberately trying to shape it? Yeah, well, look, I think there's quite a few things to say about this. First of all, we haven't, despite thousands of years of thinking about this, there isn't an agreed upon definition of consciousness. So that's one issue.

20:23

I think, and that's one issue with labeling things conscious, especially the hard problem of consciousness, you know, is it really feeling this internally like the way we seem to? You know, behaviorally these things... Well, I don't know about you, but... Yes, exactly. But the behaviorally... Well, but I used to argue with Daniel Dennett about this, that there's an additional thing is we're not just behaving in ways like we'd imagine conscious beings behave, but we also share the same substrate, where we have the same squishy brains.

20:48

And so then it's most parsimonious to think you're feeling what I'm feeling. But of course, with an AI, that's not the case. You don't have the second, the substrate equivalence. So I think it's going to be very hard to conclude that. I think there are dangers. Look, there's two reasons why I think we should be focusing on building intelligent tools today, not conscious entities.

21:07

I think for one thing is a safety argument and technical safety and risk, which is that there's a direct trade-off between creating something let's say, with self-volition, right, or self-awareness, even prior to consciousness, these are the prerequisites, right, to a conscious entity, that it will, and that and instruction following, or sometimes called corrigibility.

21:29

So this idea of a system actually following instructions, and if it's going wrong, you telling it, you know, you as the user, or the human user telling it, like, stop doing that, right, or switch yourself off, or whatever it is that you're going to, you So if you are training your system to be able to reject commands based on its own internal workings, then you're setting up a situation where if you've got your constitution wrong or your rules wrong, and this is what Asimov was always warning about with his three laws, that was a very simple form of a constitution, and his books are all warnings that you're never going to get that right in every corner case, in every edge

22:09

case. And I think his books show that very well. And yet we're sort of, I think, a bit hubristically thinking that that, some people thinking that, you know, sometimes called the misalignment problem is sort of under control. Just because today's systems, you know, the good news is they seem more malleable than we might have imagined 15, 20 years ago.

22:26

which is great news in terms of their steerability. But that doesn't mean we've solved the misalignment problem in total as we get closer to a GI and superintelligence and so on. And I just think it's dangerous to think one has. So that's the first thing. There's a technical problem, especially if you combine it with something like an RSI.

22:43

Yes, exactly. Exactly. It's the reason why we're going in this direction. Right, so RSI with self-volition is, you know, sort of doubly dangerous. So that's one thing. Second thing is a societal, more of a societal issue, which is that I don't think we are ready yet as a society. We haven't had the debate with a wider society, philosophers, theologians, artists, humanities, not just technologists, as to, you know, and from many different cultures, like, do we want to cross that second Rubicon? Are we ready to have the discussion about do these systems have moral standing? Do we need to worry about what are we doing with them in this box? Are they slaves of some sort? All

23:21

these questions come up that are complex questions, and I don't think we're equipped to even begin to start dealing with those questions. So my view is we should cross the first Rubicon on its own, which is build these intelligent tools and then actually use these intelligent tools to do more neuroscience, to go back and do more neuroscience to actually understand what the neural correlates of consciousness really are.

23:46

Finally, understand that. And then, you know, that'll probably take five, 10 years and then cross the second Rubicon. Well, to have a discussion about should we cross the second Rubicon and then if so, how, right? Of the conscious entity type. And how would you, because you engaged in hours of these discussions, how would you answer the folks who say, look, it's the only way we get the intelligent correlate to be powerful enough is it has, as it were, the ephemeral accidental.

24:12

Yes. It actually has this cognitive state and so forth. So we should talk about this a lot. We discussed this, you know, even 10 plus years ago, and I discussed it in my PhD back 20 plus years ago of, look, I felt and I still feel intelligence and consciousness are double dissociable, which means that you can have an intelligent system that's very smart on IQ and so on, kind of like the systems we have today, which I don't, I personally don't feel are at all conscious yet, right from my own, you know, everyone probably has their own view, but I don't feel yet.

24:40

And then you can have conscious entities that are not that smart. For example, my pet dog, it's lovely, and he's pretty conscious, I think. He dreams and everything, but he's not that smart. And that's provable. So I think you can have double dissociation here. Many of the higher animals, dolphins, monkeys, and so on, elephants, I think most of us would view that they have some, you know, they're conscious beings, and they exhibit all of that sort of behavior.

25:07

And of course, they're also made of carbons and the same neurons and types of things we're made of. So I feel like it's pretty obvious they have some of those qualities, but they're not necessarily that smart. So it feels like you can dissociate these things. And I think to the extent that you can, I feel like we should. You know, we're coming up close to the end of our session and the end of the summit, sad.

25:29

But What should we think about in terms of the whole notion of AI sovereignty? Because obviously one of the things that is kind of going around in a lot of countries right now is, oh, is this really important? And one of our earliest conversations was with you trying to solve this problem. So I wanted to share some thinking, and obviously it's updated for whatever that was 13, 14 years ago, but...

25:53

Yeah. Well, look, I think first of all, first thing to say is it looks to me like there's going to be, you know, half a dozen, six to eight, if you include Chinese companies, company, you know, offerings at the frontier or near the frontier. And it feels like there's going to be leapfrogging and maybe something's like three to six months ahead, but it's going to be relatively close.

26:12

So there's a lot of options. It depends if you're talking about an enterprise or a country, right? So I think as an enterprise, you have a lot of options to arbitrage between these different things. Some of them are better at certain things, you know, multimodal understanding versus coding, or it's very hard to be better on all vectors of importance at once.

26:32

So I think that's going to happen. And then open source, which also has its risks, because how do you, as we've discussed for many, many years, how do you restrict bad actors using that whilst enabling, of course, as many good actors, scientists and others to use these fantastic tools? So that's an unresolved problem. But open source is maybe only lagging six months to a year behind.

26:52

Obviously, we need kind of Western open source solutions. But I think at this point, if we're talking about countries as well, all companies that need their data to be private or they want to fine-tune further on their own, sensitive data, again, be that enterprise data or sovereign data, I think that's going to be one solution. There's also looking at what are the downstream things that can be done with these tools, big companies and big sectors that can benefit from these tools like healthcare, that you can just use this almost like commodities and then build the value on top of it.

27:25

rather than, you know, I wouldn't recommend every country, for example, trying to build a frontier model. That just doesn't make sense to me. So we're going to have to find some other way of shortcutting that. Yes. Well, and also collaborating and working out arrangements with multinational companies like your own. I think one of the reasons I wanted to kind of end on that question is that part of the theme of the summit is agency.

27:50

And And agency is not just a question of individual agency, like how do you take agency, but it's also a collective. How do we take agency? And I think part of the thing that's actually important in the AI age is that we take agency not just as any one country, but as countries together. It's part of how we have a we are in this together and we are building for the benefit of humanity.

28:10

And I think that's the kind of arc of we start with Sarah Fryer and thank you for joining us because ending endemic has been... with such intensity that I was so grateful that you set up the agenda for me thinking about how to help in this from our very first, you know, discussion at King's Cross. So let's give Demis a hand. Thank you. Thank you.

28:33

Thanks again to Demis Hassabis for joining us at the 2026 Masters of Scale Summit. This conversation was recorded live on stage in San Francisco at the amazing Presidio Theater. I'm Jeff Berman. Thanks for listening.

28:58

Humans will never be more intelligent than AI. There's going to be two types of companies. Those are great at AI and those that went out of business because they weren't. How do we build a future that is human-centered? I'm Rana El-Khayoubi, and on my podcast, Pioneers of AI, we answer that question and so many more. As an AI scientist, entrepreneur, and investor, I know what it takes to build AI that works for everyone.

29:27

Every week, I sit down with the pioneers shaping our future, and we take you behind the scenes of the AI that's transforming our lives.

29:36

Find Pioneers of AI wherever you tune in.

29:42

Masters of Scale is a Wait What original. Our executive producer is Eve Tro. Senior supervising producer is Trisha Bobita. Associate producer is Masha Makatanina. Video editor is MG Frederick. Senior talent executive is Stephanie Stern. Mixing and mastering by the audio boys themselves, Aaron Bessanelli and Brian Pugh with original music by the remarkable Ryan Holiday.

30:07

Visit mastersofscale.com to find the transcript for this episode and to subscribe to our newsletter. And be sure to subscribe to our YouTube channel.