What Has Stayed the Same Since Google's First Tensor Processing Unit?

Amin Vahdat says the tensor processing unit, at a medium level of detail, has not really changed since the first version. A CPU instruction set is the comparison: loads, stores, adds, subtracts, and branches stayed in place while the software on top changed completely. The tensor processing unit has the same kind of stable base. The instruction set has been extended, but the primitives remain: numerics specialized for very large matrix-multiply units, a sparse core for vector operations and scatter-gather, and a remote load and store that reads and writes memory across the chip interconnect. About five or six things define it.

The primitives have carried many generations of models and many generations of deep neural network algorithms. Predicting the most common workload five years out looks impossible. The stable primitives are why a multi-year hardware plan can still be written.