Leopold Aschenbrenner

Situational Awareness sold basically its entire publicly traded portfolio to the hedge fund Citadel at a 10 percent discount after a July selloff in artificial-intelligence shares left the fund down 67 percent.

Gearing amplified the loss. Many investors now use geared exchange-traded funds, and in countries like South Korea and the US some use single-share geared funds. Mark LaMonica says gearing works both ways and amplifies returns on the upside and on the downside. Situational Awareness needed to unload positions fast, because selling the public shares on the open market would have caused further falls. The fund first considered selling a prize stake in Anthropic, which is supposed to go

Capital still matters after superintelligence because the decade after artificial general intelligence stays wild, and Leopold Aschenbrenner says there is a lot of money to be made if the bet is done right. Dwarkesh Patel asks why start an investment firm at all when artificial general intelligence is only a few years away. Leopold Aschenbrenner says the screen does not go blank when artificial general intelligence or superintelligence happens.

People underrate the decade after the intelligence explosion. Leopold Aschenbrenner says the explosion is maybe the most wild period, and he says the decade after will be wild too: human institutions next to superintelligence, geopolitics gone crazy,

Mormon fertility rates fell once the community grew big enough to live in close contact with ordinary society, Leopold Aschenbrenner says. High-fertility religious subgroups normalize, and the force of modernity pulls the birth rate down.

People had treated Mormons as the group that would keep a high birth rate and replace everyone else. Aschenbrenner says the decline still reached them, and that is what clinched the broader fertility-decline story. He does not remember the exact numbers. He puts the drop at roughly four children to two over the course of 10 or 20 years. Fertility rates among Mormons remain important, and they are still going down.

The same limit applies to the next group

Leopold Aschenbrenner published knowing people in the PRC would read the series. He calls it a tough tradeoff he thought about a lot, and he says waking people in the United States, the Western world, and the Democratic coalition is ultimately imperative.

The cat is already out of the bag. AGI is not new anymore, a lot of the takes are old, and he held similar views a year ago. He might not have written it up a year ago, in part because the cat was not out of the bag enough. Managing the challenge, he says, means much broader swaths of society will need to wake up. Getting the project takes a broad bipartisan understanding of the challenges facing the country. His hope is that more people i

Leopold Aschenbrenner says China is impenetrable to him. Reading German newspapers gives him the German debate and the German state of mind, and he would understand both far more poorly from afar. He also has a sense of what is happening in the UK. He does not have a sense of the state of mind or the political debate of an average Chinese person or an average Chinese elite. China has a billion people, and he finds the distance worrying.

Almost everything else is globalized, including the internet, and he says translation machines are now basically perfect. China is still impenetrable. Some people do really great work going through party documents and party speeches. Leopold Aschenbrenner sa

Leopold Aschenbrenner says the private world is extremely tough for alignment, even when labs nominally care. A race among commercial labs leaves no headroom to hold back for three months and get alignment right, or to dedicate 90% of compute to automated alignment research. China has the secrets and sits right on their tails, so a private race is a fever struggle with no room to maneuver.

Getting alignment right during the intelligence explosion requires room to maneuver and a clear lead. Without a clear lead, he says, the spot is incredibly tough.

He has some faith in the ordinary mechanisms of a liberal society. The science will develop, alignment measurements will get better, and the c

Leopold Aschenbrenner says an AI research explosion can start before robots do ordinary physical work. AI research does not need anyone to plow a field or dig up coal, so it skips the real-world bottlenecks that slow other jobs. A system that cannot flip a burger can still make algorithmic progress, and algorithmic progress is what later figures out a robot that flips the burger.

He calls 100 million copies a lower bound we can definitely run, and says one of the first projects is turning quantity into quality. Even at the baseline pace, four years separate the preschooler from the high schooler, and smarter systems arrive quickly. Simple algorithmic changes, he says, are likely: one Alec R

Leopold Aschenbrenner says ideas get harder to find because each next discovery takes a log-scale jump in cumulative research effort. On a log-log plot the pattern is a straight line, with log cumulative research effort on the horizontal axis and log output on the vertical axis. The first discovery is easy. Later discoveries show up only after total research effort has multiplied again.

Dwarkesh Patel cites the point Patrick Collison and others have made. Compared with 100 years ago, the world has 100 times more researchers, and progress is not happening 100 times faster. Leopold Aschenbrenner says the hundred-times figure is easy to inflate and the increase is probably at least 10 times. A

Leopold Aschenbrenner says the drama at OpenAI comes from a real belief that OpenAI is building AGI, then a failure to take the implications seriously or keep the commitments that belief produced.

There would be a lot less drama if OpenAI claimed only to be building ChatGPT or business software. The AGI claim is not for marketing. A report about raising $7 trillion for chips only makes sense if OpenAI really believes in AGI. Aschenbrenner says the technology will be incredibly powerful for good and for bad. National security follows from that power: protecting secrets from the CCP, and whether America or a Middle Eastern dictator controls the core AGI infrastructure. OpenAI committed 20% of

Leopold Aschenbrenner says OpenAI told employees he was fired for leaking. The leak was a brainstorming document on preparedness that he shared with three external researchers for feedback.

He wrote it sometime last year, on safety and security measures needed in the future on the path to AGI. Sharing safety ideas with external researchers for feedback was totally normal at OpenAI at the time, and it happened all the time. The document was his own ideas. Before he shared it, he reviewed it for anything sensitive. The internal version had a reference to a future cluster, but he redacted that for the external copy. The document linked to his internal slides, but the link was dead for the exte

The Future Fund collapsed from one day to the next in November of 22, after it came out that the backer was a giant fraud. Leopold Aschenbrenner says the collapse was devastating for people with money on FTX and, closer to home, for the grantees.

Dwarkesh Patel notes that Leopold, at 18, was on a team of four people in charge of deploying billions of dollars. Leopold calls that stretch the heyday. He had worked 70 hour weeks every week for basically a year, on a tiny team that felt like a startup, to build the fund up. Then it was all gone, and the work was tied to the fraud. The team had wanted to help grantees it thought were doing amazing projects. Instead it left them with a giant probl

Leopold Aschenbrenner says alignment during an intelligence explosion looks like a war in a fog of war, not years of bureaucracy that end in written standards. He says startups in a commercial race are not fit to handle weapons of mass destruction.

Usual safety talk means a body like NIST spending years on expert consensus and then writing guidelines. Leopold Aschenbrenner says the intelligence explosion will not wait for that. The live question is whether it is safe to take the next order of magnitude. Three orders of magnitude in, the people running it no longer really understand what is going on, and a bunch of generalization scaling curves look not great. Automated AI researchers doing

An early nationalization is what would lock stuff down in time, and Leopold Aschenbrenner expects America to move only once events are already obvious, in a late and chaotic crash program like COVID. Dwarkesh Patel asks why rush, because nationalization is a one-way function. He grants that Leopold Aschenbrenner may have a point if takeoff is extremely fast: one year from AGI to ASI, then years of extraordinary technological progress. In a slower world, Dwarkesh Patel wants the project kept private until there is more evidence.

Leopold Aschenbrenner says the country will not nationalize tomorrow and will not nationalize while it is still unclear what is happening. He rejects the comfort tha

Dwarkesh Patel says Leopold Aschenbrenner told an influential person he is not going to spend any money on consumption until he is ready to buy galaxies.

Leopold then went to the restroom, and the group had a real debate. One side did not know whether he meant Galaxy, the private plane brand, or the physical galaxies. People who knew him said he meant the galaxies. When Leopold came back, someone said they were debating whether he meant buying the galaxy or the other thing. Leopold assumed the split was whether he wants to buy the property rights of the galaxy or actually just send out the probes right now.

Leopold Aschenbrenner says an arms control agreement is very unstable when the race is close, because three months ahead is enough to win.

He says the incentive to race ahead and break out is enormous. The plan is to run the intelligence explosion, and if one side gets three months ahead, that side wins. Any arms control agreement reached while the race is close is, he says, very unstable. Dwarkesh Patel set the claim beside an argument for nuclear disarmament he had debated on his podcast. In that argument, a country that breaks out and starts developing nuclear weapons would leave about six months, enough time to build international consensus, invade, and stop the country from getting nuc

Leopold Aschenbrenner says a neck-and-neck race with only a three-month lead is incredibly dangerous, while a 20 or 30 year technological lead is totally decisive, as in Gulf War One. He says a year or a couple of years after superintelligence might amount to decades of progress once the systems hit research and development and an industrial explosion with robots.

If the other side steals the weights and the algorithms and stays close, pulling ahead by three months still leaves a fever struggle. If they get ahead, they get to dominate, and Aschenbrenner says maybe they would get a decisive advantage. They build clusters like crazy and throw caution to the wind, so the leader has to keep up.

Leopold Aschenbrenner says stealing important unreleased AI code is extremely easy. DeepMind does not claim that taking the code is hard. DeepMind laid out security levels from zero to four. Level four is the level resistant to state actors, and DeepMind says the lab is at level zero.

A recent indictment described a man who stole a bunch of really important AI code and went to China with it. Leopold Aschenbrenner says the man only had to copy the code into Apple Notes and export it as a PDF, and the export got past monitoring. He says Google probably has the best security of any of the AI labs because of the Google infrastructure. He says he does not know the precise picture, and he still c

Leopold Aschenbrenner says a narrow coalition of democracies should be the group developing artificial general intelligence. A broader coalition can include dictatorships, and that broader group would get some of the benefits of AI: products, meta-recommendation engines, and last-generation models. By default the United Arab Emirates would not hold a core seat at the artificial general intelligence table.

He says he does not know the AI labs' plan for the landscape. One thing the Middle East does offer is capital, but America has plenty of capital: trillion-dollar companies and very deep financial markets. Microsoft could issue hundreds of billions of dollars of bonds and pay for the cluste

Leopold Aschenbrenner says a 100 gigawatt AI cluster uses over 20 percent of United States electricity production. He calls 100 gigawatts pretty wild, and he says the scale is still pretty doable with natural gas.

A 10 gigawatt training cluster is quite doable, at a few percent of United States natural gas production. Inference on top of the training cluster uses more power and pushes the total toward 100 gigawatts. The easy way to get power would be buying an aluminum smelting plant and replacing the plant with a data center. Long-term power contracts are locked in, and people resist the switch, so companies are building new power.

Aschenbrenner says the shift gets really interesting when

Leopold Aschenbrenner says his great-grandmother lived through the Nazi era, World War II, and most of her life under the East German communist dictatorship. She was almost 60 the first time she lived in a free, wealthy country.

She was born in 1934. As a child she saw the firebombing of Dresden from a country cottage. In 1954, during a popular uprising, Soviet tanks came in, and her husband told her to get off the streets and get home quickly. A son tried to ride a motorcycle across and was put in a Stasi prison for a while. Aschenbrenner's mother grew up in the former East and his father in the former West. They met shortly after the wall fell. He says the end of the Cold War is the reaso