Walter Goodwin
Related people
Episodes
Notes
No Priors: Artificial Intelligence | Technology | Startups · Frontier Chips for Frontier AI Labs, with Walter Goodwin…
-
How Does Model Architecture Change Chip Design?
Walter Goodwin says chip design has to follow the model architectures already on the way, and a chip with a new limit can pull later architectures toward it. Models today are trained on high-bandwidth-memory GPUs and XPUs, so a useful accelerator still has to run today's autoregressive mixture-of-experts transformers faster. Fractile's planning bet is memory bandwidth, about 25 times more per chip than an HBM-based chip.
That extra bandwidth changes which designs are practical to serve. A sparser mixture of experts saves flops for the same intelligence, from about one expert in 16 toward one in 128 or one in 256, but those models are often bandwidth-prohibitive on today's HBM chips and get
-
Why Do AI Labs Run Many Chip Platforms?
Walter Goodwin says labs deploying AI at scale try to run as many separate chip platforms as they can, because compute is existential and they need a diversity of supply. A joke in the market says first-party chip efforts exist mainly to reduce the price people pay Nvidia, and there may be some truth in the joke.
In-house chips are architecturally quite similar to what labs can already buy. The bet does not enable a fundamental capability other chips cannot enable. It is gamesmanship: build some silicon internally, or buy from AMD, to get a better price from Nvidia, while also adding aggregate capacity and control.
Chips that give new capabilities are a different category. Everybody who wa
-
Why Do AI Chips Need a Three Year Life?
Walter Goodwin says the chip that actually gets made still needs a useful lifespan of three years or more, so architectural bets have to stay thoughtful even while the design cycle is compressed.
Front-end design work can take 12 months, and cutting it to a tiny fraction brings a limited return if the next piece keeps the conventional pace. Foundry partners all share the same fab cycle times: from sending the chip until it comes back is three to five months, even in a super hot lot. After the chip returns, a sensible financial decision requires a three to five year amortization window. Huge value still sits in compressing the overall chip design cycle, because a shorter cycle means more sho
-
Why Does Fast AI Inference Come Down to Memory Cost?
Walter Goodwin says inference at data center scale, for thousands of users, collapses to the cost per gigabyte of the memory in use. The real elevator on capability is a multi-trillion parameter model running comfortably at many thousands of tokens per second, because that speed makes very long-running agents radically faster. A snappier chatbot is only the faster-horse version of fast inference.
Fast inference chips already have super high bandwidth memory, but the capacity is incredibly low. In deployment they are not running long context attention, so the job still flips back to a GPU. The gap is sharpest where the models could get much faster, and the technical capability is still missi
-
How Does Fractile Build AI Chips in House?
Walter Goodwin says Fractile builds AI chips in house with about 150 people who cover workload understanding, front-end design, physical design, back-end implementation, and advanced packaging. The group stays skinny in each area and runs one agile closed loop instead of handing a finished stage to an outside partner.
The loop starts by running ahead of the chip. The team looks at where a model architecture should change so it lines up with the Fractile bet, and judges the scaling-law direction of the bets being taken. They then advocate for the direction with customers and partners. The same work informs a set of bets for a chip that is not necessarily going to exist in volume for one or t
-
What Does Fractile Build?
Walter Goodwin founded Fractile in the summer of 2022 to build very fast inference chips for the world's largest models. The bet on speed was in place from the outset.
Foundation models trained on the internet were arriving and generalizing across everything, while a few people argued for pouring more compute into models at test time. AlphaGo is the picture: a capable neural network becomes superhuman when rolled out at scale, and the same idea had to reach language modeling. Over the past four years the pursuit has been chips that run huge models much faster than the chips of today, in a form that scales to models beyond the frontier today and to extraordinarily long contexts.