Smash Notes

Back to Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile, with Walter Goodwin.

How Does Fractile Build AI Chips in House?

Walter Goodwin says Fractile builds AI chips in house with about 150 people who cover workload understanding, front-end design, physical design, back-end implementation, and advanced packaging. The group stays skinny in each area and runs one agile closed loop instead of handing a finished stage to an outside partner.

The loop starts by running ahead of the chip. The team looks at where a model architecture should change so it lines up with the Fractile bet, and judges the scaling-law direction of the bets being taken. They then advocate for the direction with customers and partners. The same work informs a set of bets for a chip that is not necessarily going to exist in volume for one or two years, and that foresight is a core muscle they have to build.

Older chip projects followed a waterfall and a handoff point. Reach a certain level, pass the work to another partner, and then live at the mercy of how the partner behaves. AI chips have to chase the tail of the workloads faster than earlier chip plays, so the bets have to land correctly and the team has to strike very fast. The mix takes a lot of skill and a lot of luck. Keeping the entire story in house is how Fractile matches the cadence the industry now sets.

Transcript

Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile ·

No Priors: Artificial Intelligence | Technology | Startups

More from this episode

  1. What Does Fractile Build?Walter Goodwin founded Fractile in the summer of 2022 to build very fast inference chips for the world's largest models.
  2. Why Does Fast AI Inference Come Down to Memory Cost?Walter Goodwin says inference at data center scale, for thousands of users, collapses to the cost per gigabyte of the memory in use.
  3. Why Do AI Chips Need a Three Year Life?Walter Goodwin says the chip that actually gets made still needs a useful lifespan of three years or more, so architectural bets have to stay thoughtful even…
  4. How Does Model Architecture Change Chip Design?Walter Goodwin says chip design has to follow the model architectures already on the way, and a chip with a new limit can pull later architectures toward it.
  5. Why Do AI Labs Run Many Chip Platforms?Walter Goodwin says labs deploying AI at scale try to run as many separate chip platforms as they can, because compute is existential and they need a…

People

More answers like this

The other side

More with Sarah Guo

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.