Back to ‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit.
Why Is AI Alignment a Science Problem?
Alignment constitutes a science problem because people do not know how to solve the alignment issue. The difficulty does not come from lack of resources or from not trying hard enough to solve the problem. People simply do not know how to align advanced artificial intelligence systems. The distinction matters because it means additional engineering effort will not resolve the fundamental gap in knowledge about how to make artificial intelligence safe.
‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit. ·
From the transcript
Which is, I don't think that alignment is an engineering problem. I think it's a science problem. It's not that we haven't got the resources or we're not trying hard enough. We don't know how. That's what the problem is with alignment. So
let's maybe start there for a minute. Can you say that at this point, and particularly over the last six months, what is being built is really dangerous? That we're dealing with things where a loss of control or some other catastrophe could be worse than a nuclear meltdown. I want to understand what it is you saw that got you to that point.
So tell me a bit about how
you came to work at OpenAI. I joined in May of 2023, the day after Sam first testified in the Senate, some months afterChatGPT first into the world. ChatGPT had been the prior November, so it had been a few months. And the company had hired someone that I know socially, Anna McConju, a friend, actually old friend of my wife's, as it happens just at random, to run the public policy function.