Why Are Big AI Model Companies a Data Problem?
Naveen Rao says the big AI model companies are largely a data problem. Most of their capabilities have not come from advances in the model itself.
The market specializes, and working on important problems creates a big market that is defensible. A power law applies to skills: humans can learn many skills, yet some skills are more valuable than others, and the same pattern holds for data and curricula. Data work started as a commodity, a volume gathering of observations. Higher-quality data sets, including higher quality for coding and for chain of thought, now carry a lot of value. A model is almost just a manifestation of what it learned.
The data layer stays important going forward. Building models and knowing how to choose data is the game right now, and that data layer does not commoditize.
