Why Did Anthropic Draw Red Lines With the Pentagon?

Dario Amodei says Anthropic drew red lines with the Pentagon on autonomous weapons for two reasons. The technology is still unpredictable and not reliable enough to be trusted with a weapon. Even if a system were reliable, a machine told to shoot one target cannot refuse an order the way a human soldier can.

He pictures an army of a million robots or a million drones with one person's finger on the button. Human soldiers are supposed to follow the Constitution and can refuse orders that break it or that seem morally abhorrent. The same refusal is not available from the models. One person controlling that many machines would undermine democracy unless the right guardrails exist, and the law has not contemplated armies of a million robots. Anthropic is not trying to stop the government. The company is saying it will not do the work. After the stand drew attention, several efforts in Congress tried to write the red lines into law, which he sees as part of how democratic change happens.

Anthropic cannot control any other company. When OpenAI said it would do the work, he asked whether the choice was a principled stand based on ethics or an opportunistic moment.