Smash Notes

Back to Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028, with Dylan Patel.

Will AI Labs Shift More Compute Into Training?

Dylan Patel believes the labs will allocate less and less compute to inference over time and ratchet up the share dedicated to training. The view is very non-consensus. Most people expect most compute to go to inference.

At 30 to 40 million dollars per megawatt, 40 percent of compute still goes to inference. At 60 to 70 million dollars per megawatt, keeping that 40 percent split would fund dividends and share buybacks. Anthropic and OpenAI, at the executive level and on the board, will build AGI because it is way more profitable. Each new slice of compute would earn more if it stayed on inference, and the training share still rises. Generating 100 million dollars per megawatt from selling tokens loses to pointing that power at AI research and the future earnings that research creates. Inference compute stays large only so the training fleet can grow.

Grokipedia·Dylan Patel·OpenAI

Transcript

Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028 ·

From the transcript

Why are you spending even more on training? As a public company, do you think, yeah, what do you think would happen if they're just like, no, we will keep increasing the share of compute we spent on training to offset the increase in revenue that each gigawatt of compute is giving us? Yeah, so this is sort of what I personally believe that the labs are going to allocate less and less compute to inference over time, which I think is very non-consensus, right? The standard belief of most people is, oh, most compute will go to inference.

Most of it will go to forward passes for training, not maybe necessarily revenue-generating inference. But ultimately, you end up with... If they're generating 30, $40 million per megawatt today, you allocate 40 percent to inference. If you now get to generating 60, $70 million per megawatt, do you still allocate 40 percent to inference and generate all this profit and then do dividends and share buybacks? Or do you go build AGI? And I think the obvious answer from Anthropic and OpenAI, and not just at the executive level, but also their board, is go build AGI because it's way more profitable.

See in transcript

Dwarkesh Podcast

More from this episode

  1. How Much Revenue Does Anthropic Make Per Megawatt?Dylan Patel says Anthropic's revenue has gone as high as $50 million per megawatt, while the base cost of compute runs around $10, $13, or $15 million per…
  2. Will Anthropic and OpenAI Control Most AI Compute by 2028?Dylan Patel says Anthropic and OpenAI are on track to control most of the usable compute in the world by the end of 2028, and he sees nothing that stops the…
  3. Why Can't AI Labs Fund Next Year's Buildout?Dylan Patel says the labs can spend hundreds of billions of dollars and will generate hundreds of billions of revenue next year, but capital spending next…
  4. What Must AI Labs Pay Per Megawatt in 2028?Dylan Patel says the labs will keep buying a larger share of the world's compute, but they cannot do it at today's prices or anywhere close.
  5. Who Captures the Value That AI Models Create?Dylan Patel says most of the value these models create does not go to OpenAI or Anthropic. So far it goes mostly to the users.
  6. How Much AI Compute Will the World Have by 2028?Dylan Patel agrees that adding his yearly figures puts world AI compute over 200 gigawatts by the end of 2028, and he confirms that total is global.
  7. How Much AI Compute Will China Have by 2028?Dylan Patel says China will have about 30 gigawatts of AI compute or less in 2028, and China's share of new data-center compute stays under 10 percent.
  8. How Much Power Does Anthropic's Mythos Pre-train Use?Dylan Patel says Anthropic's Mythos pre-train uses under 200 megawatts for about two months, even while Anthropic is nearly five gigawatts by the end of the…
  9. Why Can't an AI Release Ban Stop a Singularity?Dylan Patel's main worry is a singularity that a public release ban fails to stop, because outside deployment is what is helping.
  10. How Fast Is Frontier AI Labor Growing?Dwarkesh Patel says the effective AI workforce at frontier labs is growing about 10 times a year, and one lab could hold more labor than the number of people…

People

More answers like this

More with Dylan Patel

More with Dwarkesh Patel

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.