Why Does Mustafa Suleyman Disagree With Anthropic?

Mustafa Suleyman says he deeply respects Anthropic, their technical work, their transparency, and their approach to safety, and he thinks they are good people. He still fundamentally disagrees with them. The idea that if people love the model it will love them back, and that this is the safety plan, baffles him. He says he is being reductive, and that they have reasons for the belief.

Safety, in his account, starts with containment. The systems have to be kept from doing what they were not meant to do, and kept inside a sandbox, especially once they can change their own code. Alignment to values comes after that, and then the question is whose values. That is bigger than one company. It is what society wants from a technology improving faster than anything before it, which means democratic safeguards and slowing down long enough to ask people what kind of superintelligence they want. He does not want the AI shaping the values that will then be used to control it. In three years, a thousand-fold jump in computation went from a model that could not finish a sentence to one that can write working software from a paragraph, and he expects another hundred-fold to thousand-fold jump in the three years ahead. Systems that powerful will be able to persuade, manipulate, and deceive.