Back to Ryan Greenblatt – What happens once AI can automate AI research?, with Ryan Greenblatt.
How Can Labs Train AI to Do AI Research?
Ryan Greenblatt says labs can train models on a whole class of containerizable, verifiable, small-scale AI research and development tasks, and skill on the small-scale tasks will transfer to extremely load-bearing aspects of AI research and development.
A model trains another AI on a small amount of compute, about eight H100s, at the scale of a GPT-2 medium or a nano GPT medium run. Reinforcement learning is the loop of tweaking and iterating on the run. The same setup covers image classification models, video generation models, image generation models, and other machine learning training tasks. The objective is training increasingly good models, or taking a particular algorithm direction and implementing it.
Companies are presumably already running reinforcement learning on small-scale research tasks. The path from there is to keep scaling the setup and keep adding more small-scale research tasks so the models keep getting better at the work.
Ryan Greenblatt – What happens once AI can automate AI research? ·
From the transcript
Yeah. Okay. So let's start with the claim that AI R&D is very verifiable. Yeah. So there's a few different parts of this. One of them is that we can train on a bunch of environments, which are like basically directly training the model to do some AI R&D task or some very close by task. So for example, we can have some environment where the model is training some AI on just like eight H100s or whatever, or like some small amount of compute.
And that model could be like, you know, the equivalent of like GPT-2 medium or whatever. And then, you know, similar to like nano GPT medium runs or whatever. And in RL, it's like tweaking and iterating on that. And we could do that for a bunch of different tasks. Like we could have it train like image classification models, video generation models, image generation models, all kinds of different sort of ML training tasks.