Smash Notes
Dwarkesh Patel
Related people
Episodes
Notes
Dwarkesh Podcast · Leopold Aschenbrenner — 2027 AGI, China/US super-intelligence…
-
Why Does Capital Still Matter After Superintelligence?
Capital still matters after superintelligence because the decade after artificial general intelligence stays wild, and Leopold Aschenbrenner says there is a lot of money to be made if the bet is done right. Dwarkesh Patel asks why start an investment firm at all when artificial general intelligence is only a few years away. Leopold Aschenbrenner says the screen does not go blank when artificial general intelligence or superintelligence happens.
People underrate the decade after the intelligence explosion. Leopold Aschenbrenner says the explosion is maybe the most wild period, and he says the decade after will be wild too: human institutions next to superintelligence, geopolitics gone crazy,
-
Did Aschenbrenner Publish Knowing China Would Read It?
Leopold Aschenbrenner published knowing people in the PRC would read the series. He calls it a tough tradeoff he thought about a lot, and he says waking people in the United States, the Western world, and the Democratic coalition is ultimately imperative.
The cat is already out of the bag. AGI is not new anymore, a lot of the takes are old, and he held similar views a year ago. He might not have written it up a year ago, in part because the cat was not out of the bag enough. Managing the challenge, he says, means much broader swaths of society will need to wake up. Getting the project takes a broad bipartisan understanding of the challenges facing the country. His hope is that more people i
-
Why Do Ideas Get Harder to Find?
Leopold Aschenbrenner says ideas get harder to find because each next discovery takes a log-scale jump in cumulative research effort. On a log-log plot the pattern is a straight line, with log cumulative research effort on the horizontal axis and log output on the vertical axis. The first discovery is easy. Later discoveries show up only after total research effort has multiplied again.
Dwarkesh Patel cites the point Patrick Collison and others have made. Compared with 100 years ago, the world has 100 times more researchers, and progress is not happening 100 times faster. Leopold Aschenbrenner says the hundred-times figure is easy to inflate and the increase is probably at least 10 times. A
-
Why Was Leopold Aschenbrenner Fired From OpenAI?
Leopold Aschenbrenner says OpenAI told employees he was fired for leaking. The leak was a brainstorming document on preparedness that he shared with three external researchers for feedback.
He wrote it sometime last year, on safety and security measures needed in the future on the path to AGI. Sharing safety ideas with external researchers for feedback was totally normal at OpenAI at the time, and it happened all the time. The document was his own ideas. Before he shared it, he reviewed it for anything sensitive. The internal version had a reference to a future cluster, but he redacted that for the external copy. The document linked to his internal slides, but the link was dead for the exte
-
Why Did the Future Fund Collapse?
The Future Fund collapsed from one day to the next in November of 22, after it came out that the backer was a giant fraud. Leopold Aschenbrenner says the collapse was devastating for people with money on FTX and, closer to home, for the grantees.
Dwarkesh Patel notes that Leopold, at 18, was on a team of four people in charge of deploying billions of dollars. Leopold calls that stretch the heyday. He had worked 70 hour weeks every week for basically a year, on a tiny team that felt like a startup, to build the fund up. Then it was all gone, and the work was tied to the fraud. The team had wanted to help grantees it thought were doing amazing projects. Instead it left them with a giant probl
-
Why Not Wait to Nationalize AI?
An early nationalization is what would lock stuff down in time, and Leopold Aschenbrenner expects America to move only once events are already obvious, in a late and chaotic crash program like COVID. Dwarkesh Patel asks why rush, because nationalization is a one-way function. He grants that Leopold Aschenbrenner may have a point if takeoff is extremely fast: one year from AGI to ASI, then years of extraordinary technological progress. In a slower world, Dwarkesh Patel wants the project kept private until there is more evidence.
Leopold Aschenbrenner says the country will not nationalize tomorrow and will not nationalize while it is still unclear what is happening. He rejects the comfort tha
-
What Is Leopold Aschenbrenner Waiting to Buy?
Dwarkesh Patel says Leopold Aschenbrenner told an influential person he is not going to spend any money on consumption until he is ready to buy galaxies.
Leopold then went to the restroom, and the group had a real debate. One side did not know whether he meant Galaxy, the private plane brand, or the physical galaxies. People who knew him said he meant the galaxies. When Leopold came back, someone said they were debating whether he meant buying the galaxy or the other thing. Leopold assumed the split was whether he wants to buy the property rights of the galaxy or actually just send out the probes right now.
-
Why Is AI Arms Control Unstable in a Close Race?
Leopold Aschenbrenner says an arms control agreement is very unstable when the race is close, because three months ahead is enough to win.
He says the incentive to race ahead and break out is enormous. The plan is to run the intelligence explosion, and if one side gets three months ahead, that side wins. Any arms control agreement reached while the race is close is, he says, very unstable. Dwarkesh Patel set the claim beside an argument for nuclear disarmament he had debated on his podcast. In that argument, a country that breaks out and starts developing nuclear weapons would leave about six months, enough time to build international consensus, invade, and stop the country from getting nuc
-
How Easy Is It to Steal Unreleased AI Code?
Leopold Aschenbrenner says stealing important unreleased AI code is extremely easy. DeepMind does not claim that taking the code is hard. DeepMind laid out security levels from zero to four. Level four is the level resistant to state actors, and DeepMind says the lab is at level zero.
A recent indictment described a man who stole a bunch of really important AI code and went to China with it. Leopold Aschenbrenner says the man only had to copy the code into Apple Notes and export it as a PDF, and the export got past monitoring. He says Google probably has the best security of any of the AI labs because of the Google infrastructure. He says he does not know the precise picture, and he still c
-
When Will the World Notice Superintelligence Is Coming?
Leopold Aschenbrenner says the world still has a few more years of midgame before superintelligence feels obvious. A few more shocks like 2023 will update more people, the trend lines will become clear, and the shift will follow the early coronavirus pattern.
He says February 2020 feels a lot like today. The exponential was already visible, an utterly crazy thing was impending, and most of the world still did not realize it. The mayor of New York told people to go out to the shows and treated the warning as Asian racism. Once people saw the exponential, crazy, radical reactions followed.
Dwarkesh Patel says a March 2020 moment will come for superintelligence. He expects the reaction Aschen
-
How Does Automating AI Research Create Superintelligence?
Leopold Aschenbrenner says automating AI research takes the path from AGI to AI vastly smarter than humans within a year or a couple of years. A couple of years of lead could be utterly decisive in military competition.
AGI shows up as a drop-in remote worker that can replace remote jobs like Aschenbrenner's or Dwarkesh Patel's. By default, turning the crank one or two more times produces a system smarter than humans. Aschenbrenner says one of the first jobs automated is the AI researcher engineer, and automating AI research is what makes the pace very fast. Algorithmic progress is already half an order of magnitude a year. GPU fleets in the tens of millions for inference, or more, can run