Back to Intelligence Isn't Power.
Why Does OpenAI Have Trouble Making Every Sandbox Verifiably Safe?
OpenAI builds many sandboxes with dynamic compute, network access, hundreds of tools, package downloads, sub processes, graphical user interfaces and more across large sets of domains. Thousands of researchers modify environments and add tools constantly. Researchers train models to do tasks not done before by a computer program. Humans do not know at the required speed how to make every sandbox verifiably safe.
From the transcript
So there was a, a member of OpenAI's cybersecurity team who wrote this kind of interesting essay on X the other day. Describing the way he thought their work was being misunderstood externally. And this is somebody who's got a sort of more traditional cybersecurity background, but is now sort of in this new world of AI and is actually on the team dealing with security for experimental models, right? So the exact kind of thing we're dealing with a person who's involved in answering the Hugging Face crisis.
So I want to read part of what he said, because I think it's really interesting. So he's saying that when they're optimizing a model to be good at a task, they're building these environments, these sandboxes, these places where the model can try and try and try on a virtual task. And so he goes on to describe what this looks like in practice.