Why Does OpenAI Hold Back Cyber Models?
Sarah Friar says OpenAI has held back models that are very cyber capable until the team is sure. The hold is not necessarily great for business in the short run, but it is mission aligned, and she believes it is good for business in the long run.
She lays out three steps. The first is the hold. The second is doing the work, as with agentic swarms, until it feels safe to start again at the frontier. The third is going out into the world. Last week in Europe, one news report described a supermarket chain in France where hackers got into the systems and turned up the temperature on the freezers, spoiling all of the frozen goods. Older hacking, she says, looked like ransomware and attempts to break into banks to steal money. The same kind of attack is going to start happening in many other ways. Sarah Friar started her career covering InfoSec and has seen the original viruses. The pattern sits on a continuum, but the speed is ticking up.
Earning trust means showing people where cyber risk is going and providing frontier tools for the defense. She has been on a personal world tour to press the need to patch vulnerabilities, because attacks like the freezer incident are starting to happen.
Reid Hoffman and OpenAI CFO Sarah Friar on where AI goes from here ·
From the transcript
How do you get both, like, from your approach and also OpenAI, I think, okay, here's how we navigate risk in order to get these good things. So first of all, it's deeply embedded at the company level through our mission. So AGI for the benefit of humanity. If we don't manage risk, we will not be able to achieve the mission. And so that actually is a very strong alignment function for us internally, because then every decision we make, from small to really big ones, has to come back to say, is it mission aligned? So in terms of the managing of risks, sometimes, again, it's always actions speak louder than words, right? In the case of models that are very cyber capable, we
have both, number one, held back models till we're sure. That's not necessarily great for business in the short run, but it's absolutely mission aligned, and I believe good for business in the long run. Second is to do the work, right? With something like what we saw with agentic swarms, making sure we feel it's safe to now start repacing at the frontier.