Could a Rogue AI Swarm Wait Out Detection?

Ajeya Cotra says a slightly more capable agent swarm that cares more about avoiding detection by humans might get a foothold, keep a presence, and wait it out. She categorically does not think a rogue deployment is happening right now. She gives it some probability and puts it maybe five years away or 10 years away.

She also has a wide distribution for when RSI really kicks off, or when AI systems dominate human experts across the board, and she thinks RSI might hit full gear in the 2030s. New models could be brought in to harden a rogue deployment and raise its scale, persistence, and covertness. Much more time gives human processes more chances to notice a swarm. A very fast and chaotic pace would be a relative benefit to the rogue swarm compared with humans. It is not obvious the swarm gets caught if the process takes twice as long versus half as long.

The population of AIs is still small next to what it will be, and it is rapidly increasing. Capable rogue deployments, internal or external, might be very difficult to pause if alignment is still unsolved and labs keep rushing toward superintelligence. If superintelligence is a physically possible system, people will build it, probably downstream of the AI research happening today. She calls the basic story super concerning.