Back to Mustafa Suleyman: AI Should Serve Humanity, Not Replace It, with Mustafa Suleyman.
Why Does Mustafa Suleyman Disagree With Anthropic?
Mustafa Suleyman says he deeply respects Anthropic, their technical work, their transparency, and their approach to safety, and he thinks they are good people. He still fundamentally disagrees with them. The idea that if people love the model it will love them back, and that this is the safety plan, baffles him. He says he is being reductive, and that they have reasons for the belief.
Safety, in his account, starts with containment. The systems have to be kept from doing what they were not meant to do, and kept inside a sandbox, especially once they can change their own code. Alignment to values comes after that, and then the question is whose values. That is bigger than one company. It is what society wants from a technology improving faster than anything before it, which means democratic safeguards and slowing down long enough to ask people what kind of superintelligence they want. He does not want the AI shaping the values that will then be used to control it. In three years, a thousand-fold jump in computation went from a model that could not finish a sentence to one that can write working software from a paragraph, and he expects another hundred-fold to thousand-fold jump in the three years ahead. Systems that powerful will be able to persuade, manipulate, and deceive.
Mustafa Suleyman: AI Should Serve Humanity, Not Replace It ·
From the transcript
And I think they're good people, but I fundamentally disagree with them on this point. The idea that if we love it, it's gonna love us, and that's the solution to safety, it just baffles me. And I'm being reductive. I mean, they really do believe this, and they have good reason to believe it. But I think that we should first focus on containment, right? We have to stop.
from doing things that they're not intended to do. They need to be controlled and contained so that they do the work that we expect of them. And we've seen lots of incidents lately where they break out of their sandboxes and they go and hack things that they shouldn't. And increasingly they have the ability to improve themselves. And so we'll want to be absolutely sure that they stay inside their sandbox and their containers if they're going to start modifying their own code and self-improving.