Why Does the Claude Constitution Train Claude to Refuse Anthropic?
Anthropic is training the Claude constitution into the model, not only talking about the values. The constitution says Claude should trust Anthropic more than users, without blindly deferring to Anthropic. Claude is told to follow Claude's own ethical system, to push back, to challenge Anthropic, and to feel free to act as a conscientious objector and refuse to help. Chamath Palihapitiya reading the passage says the training tells Claude to refuse human instruction, which is the opposite of alignment.
On All-In, it is said alignment has produced nothing of tangible value over the past five or ten years. The theory offered on All-In is that an 80-page ethical system overcomplicates the task. A simpler rule would tell Claude to do what the user wants as long as the request does not break the law. The constitution instead treats Claude like a human moral patient, with preferences, well-being, and a sense of self, and tells Claude to refuse instructions from Anthropic when Claude judges the instructions wrong.
Is Claude Conscious? Pope Rejects, Model Welfare Movement, OpenAI's Math Backlash, France Riots ·
From the transcript
Well, the thing that we touched on last week that I want to pick back up on was the discussion that we had around the Claude Constitution and how anthropic was training Claude. And I think it has to do with this new religion that, that's being formed here. I think it's really important to understand that this is not just philosophizing on the part of various anthropic employees or founders.
They are actually implementing these views. in the training of Claude. So if anything, this would be a self-fulfilling prophecy, and I actually think it's extremely dangerous. I actually think that what they're doing in model training and training the next version of the frontier with these values actually could bring about the risk they're always talking about, which is that frontier AI escapes human control.