Why Is AI With No Guardrails Dangerous?

Nikesh Arora says AI with no guardrails is dangerous because safety depends on the weakest link, and a company that gives out the raw power of AI with no controls hands a nuclear bomb to anybody who wants it.

He points to debates in the White House on responsibility, alignment, and testing models beforehand. At one level, he says, people want global treaties to manage the pacing and development of AI for good. At the same time, a company can still offer the most powerful tool with no control, no guardrails, and no governance. He compares the offer to a do-it-yourself aspirin kit for building medication, or to handing someone powerful tools, and says he does not understand the business model. In his words, the company is inducing product liability as a service. On All-In, the reply is that the service goes to white hats under very constrained terms.