Smash Notes

Back to Inside Anthropic’s trillion-dollar AI gamble, with Joey Brookhart.

How Did Anthropic's Inference Margins Turn Positive?

Joey Brookhart says Anthropic's gross margin on inference went from negative 94% two years ago to positive 85% now. The unit he uses is revenue per megawatt of inference. SemiAnalysis puts Anthropic at about $65 million of revenue per inference megawatt, up from the mid-teens a year ago, and an Opus 5.5-class model on Blackwell could reach $100 million to $120 million. Compute cost is a little over $10 million per megawatt. Token throughput on Amazon Trainium chips is up 5 to 10 times year over year, from the hardware and from how the model runs on it.

Only about 30% of the compute is serving paying inference. The rest is experimentation, research, and training the next model, which is where the returns and the defensibility sit. Nilay Patel expected the S-1 to show current models throwing off margin that would be poured into training, because Anthropic is chasing AGI and the filing says Claude might be conscious. Training cost is instead managed against the inference margin. With gross margins of 85% or better, the lab can raise the share of compute that goes to training. Growth came in faster than planned. An 11% non-GAAP EBIT margin was not the goal. They would rather sit closer to zero, or even below, if that compute went to training. The roadshow has pointed investors at the profitability curve and at how little cash was burned to get here.

Transcript

Inside Anthropic’s trillion-dollar AI gamble ·

From the transcript

And I think for you know, investors or just people that are interested in the business, it's, you know, Anthropic had negative 94% gross margins on inference two years ago. Now they have positive 85 and people didn't understand kind of, you know, how do they allocate compute? What are actually, you know, how do we think about the inference business model? I think, you know, part of that gross margin conversation is the revenue per megawatt in inference.

And why are all companies that are running inference, uh, um, seeing massive gains in, you know, inference and compute efficiency. So we try to, you know, model out these businesses and, you know, obviously when Anthropic was private, you know, we're dealing with information we can get from, you know, the press and things like that. But, uh, you know, now that we have numbers, um, Concrete audited numbers.

See in transcript

Decoder with Nilay Patel

More from this episode

  1. What Is Anthropic's Annual Recurring Revenue?Joey Brookhart says Anthropic's S-1 shows a company that scaled, with capital efficiency, to a valuation above a trillion dollars and more than $70 billion in…
  2. Why Does 80% of Anthropic's Revenue Come From New Models?Joey Brookhart says 80% of Anthropic's revenue in the last three months came from models released in the last six months.
  3. Does the AI Industry Follow Apple's iPhone Playbook?Nilay Patel says the AI industry does not follow Apple's iPhone playbook.
  4. Why Does Anthropic's S-1 Show an $80 Billion Loss?Joey Brookhart says Anthropic's $80 billion loss is a non-cash charge for warrants and convertible notes after the valuation jumped, not cash missing from the…
  5. Can Anthropic Pay for Its Compute Commitments?Joey Brookhart says Anthropic can cover its compute commitments with revenue. Nilay Patel cites an analysis above $800 billion over 10 years.
  6. How Do OpenAI's Margins Compare With Anthropic's?Joey Brookhart says OpenAI's margins do not look like Anthropic's.
  7. Why Did Anthropic's Third Quarter Slow Down?Joey Brookhart says Anthropic's third quarter slowed because Meta, probably more than 10% of revenue, cut Claude Code seats from 60,000 to 30,000, and because…

People

More answers like this

More with Nilay Patel

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.