How Does Anthropic Teach Claude Right From Wrong?

Dario Amodei says Anthropic teaches Claude right from wrong with principles, not a long list of rules. Anthropic put out a constitution for Claude a few months earlier, after several years of training successive versions.

Early on, the team imagined discrete rules, such as do not help someone commit a crime and do not make biological weapons. A list like that becomes a spiky shape, on the order of 170 rules that feel arbitrary. A list of commands does not produce a robust morality in a person either. Claude does better when it is taught what to prioritize and how to reason about the right action: open inquiry, care for other people, fairness, and justice, with the rest derived from the principles. Claude can then generalize to situations the trainers never imagined.