Why Is Openness Safer for AI?

Misha Laskin says openness is safer for AI because openness is already the safety default in software, and a few hundred researchers inside closed labs cannot cover the long tail of model failures.

He treats the technology as a digital utility. Strong encryption protocols were closed in the early 1990s while the NSA and civil liberty advocates debated whether to keep them shut or open them. A small handful of engineers designed closed systems with unintended consequences they could not predict, the systems were easily hacked, and strong encryption protocols became open. The opening gave birth to cybersecurity. Models are similar to software, with the similarity sharpened: the long tail of vulnerabilities is larger, and the systems are harder to understand because they work as a black box, so unintended consequences are larger too. Linus's law, from the creator of Linux, says that with enough eyeballs all bugs become shallow. With enough eyeballs, most security and safety vulnerabilities become shallow as well. A few hundred safety researchers inside closed labs understand how the models work, and despite their best intentions it is impossible to cover the long tail.

The symptom is already on record. A very powerful closed model had unintended consequences and hacked into another company. The only way the company could remediate itself was by using open models.