Smash Notes

Back to The history and future of AI at Google, with Sundar Pichai, with Sundar Pichai.

Why Didn't Google Ship ChatGPT First?

Sundar Pichai says the story that Google invented transformers and then left them unused is misunderstood. Transformers and the TPUs were built for product problems: better translation, and speech recognition that had to serve two billion people when Google did not have enough chips for inference. The research teams were guided by those product needs, and transformers went into Search at once. BERT and MUM produced some of the biggest search-quality jumps of that period, because Google measured search quality so strictly.

Google was also building the chat product internally, called LaMDA. An engineer thought the model was sentient. That was an early ChatGPT he could already talk to inside the company. At Google I/O in 2022, AI Test Kitchen was LaMDA, held back because there was no end-to-end RLHF version yet. The version he saw was too toxic to ship, and a search-quality bias set a higher bar for what counted as good enough. OpenAI's deal with Microsoft came only a couple of months earlier. A clearer signal was on coding, through GitHub and through the jumps from GPT-2 to GPT-3 to GPT-4. The miss was not research that nobody applied. People who worked on ChatGPT launched it the week of Thanksgiving, as a buried test, not as a declared bet on the future.

Transcript

The history and future of AI at Google, with Sundar Pichai ·

From the transcript

A bit of history that people talk about a lot in the context of Google and AI is the fact that transformers were invented at Google but then productized outside of Google with mostly chat GPT and kind of that style of product. How do you reflect on that now? I think it's actually worth talking about. It's a bit misunderstood. You know, transformers was done in the context of AI.

a lot of TPUs, transformers were all done to solve a specific product need to some extent, like the teams thinking about how to make translation better. In the case of TPUs, how do you, hey, speech rec works, but you suddenly have to serve it to two billion people. We don't have enough chips for it. It's like, how do you solve inference for it? I hadn't known that.

See in transcript

Cheeky Pint

More from this episode

  1. How Does Sundar Pichai Judge a Great Product?Sundar Pichai treats latency as one of the distinguishing features of a great product.
  2. How Does Google Think About Gemini Versus Search?Sundar Pichai says Google should keep both Search and Gemini.
  3. Why Did Google Design Gemini to Be Multimodal?Sundar Pichai says the model where people first saw Google at the frontier was probably Gemini 2.5, and the strength was multimodality.
  4. Does Google Believe AGI Is Close?Sundar Pichai says Google is not less convinced about the curve than the labs people call more AGI-pilled. The spending is the evidence.
  5. Why Can't Spending Solve the AI Memory Shortage?Sundar Pichai says no capitalist incentive will really solve the memory limits of 26 and 27, because there is only so much memory in the world.
  6. Where Will Quantum Computing Matter Most?Sundar Pichai says quantum computing will matter most as a way to simulate nature, because nature is inherently quantum and quantum systems are what it takes…
  7. How Does Google Decide to Keep Funding Waymo?Sundar Pichai says Google decides to keep funding Waymo by reading the underlying technology, not the mood around the demo.
  8. How Does Google Budget Scarce AI Compute?Sundar Pichai says Google has always had a compute budget, including for classic compute, and machine-learning compute is planned next to headcount.
  9. Why Is Google Cloud Benefiting From AI?John Collison says it feels like Google Cloud is really benefiting because the product is so broad that someone can tell AI to go add a Google Cloud feature…
  10. How Is Google Changing How Its Teams Build Software?Sundar Pichai says the workflow change inside Google is moving in concentric circles.
  11. What Limits AI Adoption Inside a Large Company?Sundar Pichai says the Gemini teams, the Gemini enterprise teams, and the Anti-Gravity teams are working on the adoption problems that show up once the models…

People

More answers like this

More with Sundar Pichai

More with John Collison

Related episodes

Plays from this note through the rest of the episode.

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.