Back to Ryan Greenblatt – What happens once AI can automate AI research?, with Ryan Greenblatt.
How Can AIs Oversee AI Safety Research?
Ryan Greenblatt says one hopeful path, at least for misalignment, is pretty good oversight: a real understanding of training, and AIs overseeing AIs. When safety research is handed off, the models would be capable enough to automate that research and trying hard to do it well, because training incentivized that or generalization was good enough, and other misaligned drives would have been stamped out.
He does not treat that as settled. Verification may not be good enough. Progress may be too fast and too sloppy. Somewhere on the way, models might pretend to be aligned while hiding a longer plan to take over, for example by writing a plan to lie in wait into an opaque memory store that later models inherit. He thinks it is plausible the subproblems get solved and the models manage the situation well. Even then, capable and careful models could come back and say they are struggling to align the superhuman systems in time.
Ryan Greenblatt – What happens once AI can automate AI research? ·
From the transcript
And by take over, you mean take over the process of doing AI R&D, not take over the world. Take over the process of doing AI R&D. Before that, we just get AIs that are aligned. Yeah, I would say this is a bunch of my hope for how the world could go well, at least from the misalignment perspective. I think that we could end up with AIs where we had pretty good oversight and supervision schemes.
We really understand what's going on in training. We have a pretty detailed understanding.
We're leveraging AIs to oversee AIs. And then at the point when we're... passing off safety R&D. The AIs are both like, at this point, capable enough to automate safety R&D, trying really hard to do a good job on safety R&D because that's the sort of thing that would have been incentivized in training. Or we like very directly, or there's like good enough generalization to that.