Smash Notes

Back to Muse Wonder: Zuckerberg and Wang's $100B AI, with Mark Zuckerberg & Alexandr Wang.

Why Is Meta Spending $100 Billion Before the Model Exists?

Alexandr Wang says frontier AI is the opposite of a lean startup. A lean startup gets something working and only then doubles down. You cannot do that here. The bets have to be made before there is proof the thing works, or you end up radically behind. He calls it entrepreneurship on hard mode, and he gives Mark Zuckerberg credit for taking those decisions.

Zuckerberg retells the end of last year. They were excited about the progress and had not seen the models yet. The check they were going to write was, as he put it, like $100 billion or $200 billion, and he asked Alex if he was sure it was going to work. John Collison describes the same constraint as pushing the chips all in before the cards are dealt, because at this scale the clusters have to be this big.

At an executive offsite in Hawaii at the end of last year, Susan pulled Wang aside more than once. She told him that if it did not work it would be really bad in financial terms, and that the model had to be good. If it was good, they would be fine. If it was bad, it would be very bad.

Transcript

Muse Wonder: Zuckerberg and Wang's $100B AI ·

Cheeky Pint

More from this episode

  1. How Does Mark Zuckerberg Use Muse?Mark Zuckerberg uses Muse to coach his mixed martial arts.
  2. Why Do Winning AI Products Look So Simple?John Collison says winning AI products look simple because the field collapses quickly onto a few designs you could describe on the back of a postcard.
  3. What Is Muse Wonder?Alexandr Wang says Muse Wonder is a new class of model at the frontier, and the lab has been working toward it since the start.
  4. What Is Active Memory in an AI Model?Alexandr Wang says Muse Wonder is state of the art, by Meta's own measurement, on a slate of personal and professional agent tasks.
  5. How Big Should a Pre-training Team Be?Mark Zuckerberg says he does not know how big a pre-training team is, and it does not need to be that big. Other labs are probably quite big.
  6. Why Does Muse Wonder Lead on Legal and Finance?Alexandr Wang says Muse Wonder is the state of the art model on a lot of the finance evaluations and a lot of the legal evaluations, and the domain skill…
  7. Why Doesn't Scaling One Part of an AI Model Work?Alexandr Wang's broad rule for AI is to scale everything, and to scale it together. Scaling one thing in isolation does not work.
  8. How Is Meta Scaling AI Past Five Gigawatts?Mark Zuckerberg says you can measure a lab's compute strength by how big its largest single supercomputer is.
  9. How Much Does Training a Frontier AI Model Cost?Mark Zuckerberg puts the figure you would see in public, to operate a gigawatt cluster for a year, on the order of $15 billion.
  10. Why Is Meta's AI Spend Less Risky Than the Price Tag?Mark Zuckerberg says Meta's huge AI spend is not as risky as the size of the numbers suggests, because Meta has a backstop for how it can use compute.
  11. How Does Muse Tell Mark Zuckerberg From His Daughter?Mark Zuckerberg says Muse voice mode segments the voices so it knows which one is him and which one is not.
  12. Why Are AI Agents Different From Chatbots?Alexandr Wang says an agent is a big step change because the mental model is different from a chatbot that only shares information.
  13. How Does Meta Plan to Pay for Its AI Buildout?John Collison asks how the AI buildout gets paid for.
  14. Why Hasn't One AI Lab Run Away With the Lead?John Collison recalls the 2023 story that one AI lab would get ahead, start recursive self-improvement, and pull away from everyone else.
  15. Why Did Researchers Join Meta Superintelligence Labs?Alexandr Wang says a reason people were excited to join Meta and Meta Superintelligence Labs is that they wanted to work somewhere focused on giving this…
  16. Why Did Mark Zuckerberg Choose Alexandr Wang?Mark Zuckerberg says he and Alexandr Wang had talked about once every few months for years, and he was always impressed by how Wang thought about strategy.
  17. Why Did Mark Zuckerberg Reboot Meta's AI Lab?John Collison names four moves he thinks even an opinionated founder would not make. Mark Zuckerberg treated the recent prior AI effort as not good enough.
  18. Why Did a Meta Lead Reject a Reasoning Model?Alexandr Wang heard a key lead on Meta's existing effort reject a reasoning model because the lead did not think consumers needed one and said a normal model…

People

More answers like this

More with John Collison

More with Mark Zuckerberg

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.