Will AI Labs Shift More Compute Into Training?
Dylan Patel believes the labs will allocate less and less compute to inference over time and ratchet up the share dedicated to training. The view is very non-consensus. Most people expect most compute to go to inference.
At 30 to 40 million dollars per megawatt, 40 percent of compute still goes to inference. At 60 to 70 million dollars per megawatt, keeping that 40 percent split would fund dividends and share buybacks. Anthropic and OpenAI, at the executive level and on the board, will build AGI because it is way more profitable. Each new slice of compute would earn more if it stayed on inference, and the training share still rises. Generating 100 million dollars per megawatt from selling tokens loses to pointing that power at AI research and the future earnings that research creates. Inference compute stays large only so the training fleet can grow.
Dylan Patel – Anthropic & OpenAI will have most of the world’s compute by 2028 ·
From the transcript
Why are you spending even more on training? As a public company, do you think, yeah, what do you think would happen if they're just like, no, we will keep increasing the share of compute we spent on training to offset the increase in revenue that each gigawatt of compute is giving us? Yeah, so this is sort of what I personally believe that the labs are going to allocate less and less compute to inference over time, which I think is very non-consensus, right? The standard belief of most people is, oh, most compute will go to inference.
Most of it will go to forward passes for training, not maybe necessarily revenue-generating inference. But ultimately, you end up with... If they're generating 30, $40 million per megawatt today, you allocate 40 percent to inference. If you now get to generating 60, $70 million per megawatt, do you still allocate 40 percent to inference and generate all this profit and then do dividends and share buybacks? Or do you go build AGI? And I think the obvious answer from Anthropic and OpenAI, and not just at the executive level, but also their board, is go build AGI because it's way more profitable.