Why Is Society Not Ready for Conscious AI?

Demis Hassabis says society is not ready for conscious AI, so the work now should stay on intelligent tools rather than conscious beings. Self-volition or self-awareness works against corrigibility, a system that follows instructions and stops when a person tells it to stop. Training a model to reject commands from its own internal rules repeats the warning in Asimov's three laws. Today's systems are more malleable than researchers expected 15 or 20 years ago. That does not mean misalignment is solved on the way to superintelligence, and combining it with recursive self-improvement makes the risk worse.

There has not been a debate, beyond technologists, with philosophers, theologians, artists, and the humanities, across many cultures, about whether these systems should have moral standing or whether they would be slaves of some sort. He does not think society is equipped to start that argument.

Build the tools first, then use them to do more neuroscience and learn the neural correlates of consciousness. He thinks that understanding probably takes five or 10 years. Only then should there be a decision about a conscious entity. Intelligence and consciousness can come apart: a system can be highly capable and, in his view, not conscious yet, and a being can be conscious without being especially smart, as with a dog or a dolphin.