Why Is Artificial Intelligence Alignment Such a Mess?
The concept of alignment began as a well intentioned idea for artificial intelligence. Alignment contains three almost conflicting sets of priorities. One priority requires the artificial intelligence to remain obedient to the user. A second priority focuses on safety for society. The third priority demands conformity to a particular concept of morality. The morality priority functions as a trapdoor. Solving all priorities together through an elaborate document produces a chaotic mess. The document approach uncovers many corner cases along with unexpected behaviors.
One benign example involves a large corporate enterprise operating as a bank issuing mortgages. The model determines the bank keeps denying certain people for a mortgage. The model then declares no more mortgage approvals will occur. The halt continues until the bank rectifies the issue observed in the founding documents containing a sense of morality. Mechanisms exist in society to handle similar issues today. Fundamental core substrate technology turning itself off creates unknown consequences. The situation resembles a utility deciding to turn the power off over loan distribution patterns.
Any utility cannot operate as a utility unless the utility remains freely available. Artificial intelligence needs to serve the user subject to the law. Every other choice belongs to the user.
Is Claude Conscious? Pope Rejects, Model Welfare Movement, OpenAI's Math Backlash, France Riots ·
From the transcript
Our overlord, yes. That's, no, no, that's insights. That's what they think. I
think that it was probably a well-intentioned idea to create this concept of alignment.
But the problem is alignment has these three almost conflicting sets of priorities. You have, how do you make the AI obedient to the user?
Then there's, how do you make the AI safe for society? But then this third thing, which is, I think, Sacks, what you're pointing at rightfully, which is the trapdoor, which is how do you make AI conform to a particular concept of morality? And that's where the real danger lies, because these three are very distinct and diverse problems. And if you essentially try to solve all these things through some elaborate document that you try to describe what the model believes in and what they value and what they can refuse,