Why Was Fable Held Back Over Security Holes?

David Heinemeier Hansson says the Fable release was held back because the model was so good at finding holes an attacker could exploit that it was not safe to ship. The same systems are unusually strong at finding those holes and at fixing them.

The hard cases are combo moves. One small vulnerability is not fatal on its own. Combine it with four others and it becomes remote command execution. People who can chain those steps are rare, and they usually work inside state-sponsored or other clandestine groups rather than publishing what they find. On that kind of security work, he thinks the models have reached a level virtually no human can match.

He still wants a closer look at the code for a Linux distribution, where speed is the obsession, and for safety-critical systems such as a nuclear plant or a self-driving car. The security result is what showed him the tools had crossed a line.