Smash Notes

Back to The history and future of AI at Google, with Sundar Pichai, with Sundar Pichai.

How Does Google Budget Scarce AI Compute?

Sundar Pichai says Google has always had a compute budget, including for classic compute, and machine-learning compute is planned next to headcount. Headcount planning stays careful too. Google uses both TPUs and GPUs, extensively. Some stretches of that compute were easy to get, and some left the company constrained. Right now the constraint is acute, so a dedicated hour a week goes to the question at a granular level. He looks at the compute units each project and each team is using. Compute is the scarce resource, and that weekly review is there so Google spends it on the initiatives worth doing.

Transcript

The history and future of AI at Google, with Sundar Pichai ·

From the transcript

Just at a very... concrete budgeting level, how does that work inside of Google? Do you have an overall TPU budget for the company, and then when you are giving a project resourcing, previously you gave us a certain headcount budget, and now you give it a headcount and a TPU budget, are they the same budget? Just how does that work when you're doing a quarterly review or an annual review? Look, we've always had a compute budget.

Ask me for a friend. Ask me for a friend. Now, we've always had a compute budget, even in classic compute.

I would say with ML, and we use both TPUs and GPUs, by the way, extensively, but ML compute planning is, we are super thoughtful about headcount planning too, but we've always had to plan that. And ML compute, we've gone through phases where they've been easy, and then there have been phases where we've been constrained as a company.

See in transcript

Cheeky Pint

More from this episode

  1. Why Didn't Google Ship ChatGPT First?Sundar Pichai says the story that Google invented transformers and then left them unused is misunderstood.
  2. How Does Sundar Pichai Judge a Great Product?Sundar Pichai treats latency as one of the distinguishing features of a great product.
  3. How Does Google Think About Gemini Versus Search?Sundar Pichai says Google should keep both Search and Gemini.
  4. Why Did Google Design Gemini to Be Multimodal?Sundar Pichai says the model where people first saw Google at the frontier was probably Gemini 2.5, and the strength was multimodality.
  5. Does Google Believe AGI Is Close?Sundar Pichai says Google is not less convinced about the curve than the labs people call more AGI-pilled. The spending is the evidence.
  6. Why Can't Spending Solve the AI Memory Shortage?Sundar Pichai says no capitalist incentive will really solve the memory limits of 26 and 27, because there is only so much memory in the world.
  7. Where Will Quantum Computing Matter Most?Sundar Pichai says quantum computing will matter most as a way to simulate nature, because nature is inherently quantum and quantum systems are what it takes…
  8. How Does Google Decide to Keep Funding Waymo?Sundar Pichai says Google decides to keep funding Waymo by reading the underlying technology, not the mood around the demo.
  9. Why Is Google Cloud Benefiting From AI?John Collison says it feels like Google Cloud is really benefiting because the product is so broad that someone can tell AI to go add a Google Cloud feature…
  10. How Is Google Changing How Its Teams Build Software?Sundar Pichai says the workflow change inside Google is moving in concentric circles.
  11. What Limits AI Adoption Inside a Large Company?Sundar Pichai says the Gemini teams, the Gemini enterprise teams, and the Anti-Gravity teams are working on the adoption problems that show up once the models…

People

More answers like this

More with Sundar Pichai

More with John Collison

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.