Smash Notes

Back to Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile, with Walter Goodwin.

Why Do AI Chips Need a Three Year Life?

Walter Goodwin says the chip that actually gets made still needs a useful lifespan of three years or more, so architectural bets have to stay thoughtful even while the design cycle is compressed.

Front-end design work can take 12 months, and cutting it to a tiny fraction brings a limited return if the next piece keeps the conventional pace. Foundry partners all share the same fab cycle times: from sending the chip until it comes back is three to five months, even in a super hot lot. After the chip returns, a sensible financial decision requires a three to five year amortization window. Huge value still sits in compressing the overall chip design cycle, because a shorter cycle means more shots on goal. A team wants to be ready to lock a flagship platform and ramp it, then live through a 12-month or 18-month volume ramp while expecting the chip to last.

Both pressures hold at the same time: keep compressing the design cycle, and still place subtle architectural bets that remain useful for three years plus.

Transcript

Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile ·

No Priors: Artificial Intelligence | Technology | Startups

More from this episode

  1. What Does Fractile Build?Walter Goodwin founded Fractile in the summer of 2022 to build very fast inference chips for the world's largest models.
  2. How Does Fractile Build AI Chips in House?Walter Goodwin says Fractile builds AI chips in house with about 150 people who cover workload understanding, front-end design, physical design, back-end…
  3. Why Does Fast AI Inference Come Down to Memory Cost?Walter Goodwin says inference at data center scale, for thousands of users, collapses to the cost per gigabyte of the memory in use.
  4. How Does Model Architecture Change Chip Design?Walter Goodwin says chip design has to follow the model architectures already on the way, and a chip with a new limit can pull later architectures toward it.
  5. Why Do AI Labs Run Many Chip Platforms?Walter Goodwin says labs deploying AI at scale try to run as many separate chip platforms as they can, because compute is existential and they need a…

People

More answers like this

More with Sarah Guo

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.