Back to The history and future of AI at Google, with Sundar Pichai, with Sundar Pichai.
How Does Sundar Pichai Judge a Great Product?
Sundar Pichai treats latency as one of the distinguishing features of a great product. A fast response almost always reflects technical underpinnings that were done well. Shipping, iteration, and release cycles are a second kind of speed, and both matter. Latency is the harder one to protect, because teams keep adding capabilities while the capability frontier keeps progressing.
On Google Search, sub-teams now carry latency budgets measured in milliseconds. Ship something that shaves off three milliseconds and the credit is 50%. The team earns 1.5 milliseconds for its latency budget, and 1.5 milliseconds gets passed on to the user.
The split is the balance. Part of a speed win can fund more capability. The rest has to reach the person waiting on the query.
The history and future of AI at Google, with Sundar Pichai ·
From the transcript
And so original Google search was really fast and famously displayed the search query time within the results sort of showing off. And then Gmail fast search compared to the competitors at the time or Chrome compared to the competitors at the time. And now, I mean, I use all of the... AI services for different things. But Gemini on TPUs is just so fast.
And I'm curious how much this is part of the explicit product strategy and how you think of it, or it's much more nuanced than that. I've always internalized speed. Let's call it as latency for this purpose, right? And as like one of the distinguishing features of a great product. And also almost always reflects the technical underpinnings of the product having been done well.