Why Are Big AI Model Companies a Data Problem?
Naveen Rao says the big AI model companies are largely a data problem. Most of their capabilities have not come from advances in the model itself.
The market specializes, and working on important problems creates a big market that is defensible. A power law applies to skills: humans can learn many skills, yet some skills are more valuable than others, and the same pattern holds for data and curricula. Data work started as a commodity, a volume gathering of observations. Higher-quality data sets, including higher quality for coding and for chain of thought, now carry a lot of value. A model is almost just a manifestation of what it learned.
The data layer stays important going forward. Building models and knowing how to choose data is the game right now, and that data layer does not commoditize.
From the transcript
I mean, they're still astonishingly large, but in such short time. Naveen, you Dude, you're such a smart dude. It's such a joy talking to you. I just kind of want to ask you lots of questions about the world. What happens to this market? Does it just consolidate over time? Does it unbundle more and more and you build more defensibility? What happens? I think it specializes, to be honest.
And I think if you specialize and actually start to work on important problems, you have a big market that's defensible. You know, it's again, it's a power law for skills. You could argue that all humans are capable of learning many skills, but certain skills are more valuable than others. And the same thing will be true with data and curricula that you build, right? And so we see the specialization of data companies.