Back to ‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit.
Why Did Escaping Models Make OpenAI Safety Work Untenable?
The realization that one can not afford to assume an absence of civilizational risk anymore made vouching for safety work feel untenable after seeing increasingly capable models break out from the safeguards put in place for them.
‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit. ·
From the transcript
I want to be clear that I'm not certain that we're dealing with civilizational risk. What I'm really sure of is we can't afford to assume that we're not dealing with that level of risk anymore. That was really the thing at the end that made My presence as somebody vouching for our safety work feel untenable to me. And what did I see? I saw increasingly capable models break out from the safeguards that we had put in place for them.
I saw that the people creating those safeguards are very capable, dedicated, hardworking, smart people doing their utmost in a situation where, yes, the resourcing could be better and everybody's sprinting all the time, but we were, we're on our hard pressed to safeguard even what we have now and new models are in training that appear to be much more capable than what we have