Back to The history and future of AI at Google, with Sundar Pichai, with Sundar Pichai.
How Does Google Budget Scarce AI Compute?
Sundar Pichai says Google has always had a compute budget, including for classic compute, and machine-learning compute is planned next to headcount. Headcount planning stays careful too. Google uses both TPUs and GPUs, extensively. Some stretches of that compute were easy to get, and some left the company constrained. Right now the constraint is acute, so a dedicated hour a week goes to the question at a granular level. He looks at the compute units each project and each team is using. Compute is the scarce resource, and that weekly review is there so Google spends it on the initiatives worth doing.
The history and future of AI at Google, with Sundar Pichai ·
From the transcript
Just at a very... concrete budgeting level, how does that work inside of Google? Do you have an overall TPU budget for the company, and then when you are giving a project resourcing, previously you gave us a certain headcount budget, and now you give it a headcount and a TPU budget, are they the same budget? Just how does that work when you're doing a quarterly review or an annual review? Look, we've always had a compute budget.
Ask me for a friend. Ask me for a friend. Now, we've always had a compute budget, even in classic compute.
I would say with ML, and we use both TPUs and GPUs, by the way, extensively, but ML compute planning is, we are super thoughtful about headcount planning too, but we've always had to plan that. And ML compute, we've gone through phases where they've been easy, and then there have been phases where we've been constrained as a company.