How Many Hours of Human Data Do Robots Need?

Josh Gao says the robot-training recipe coming together looks like a million hours of human data across egocentric capture and other modalities, tens of thousands of hours of UMI data, and a couple of hours of last-mile teleoperation plus in-domain, in-distribution demonstration.

A year ago, many people argued that only UMI data would work, or that only egocentric data would work. The streams sit in one pot of soup, and every ingredient matters at some point. He does not subscribe to a single data religion, because tactile data has not entered the mix yet. Learning how much force to apply to a door handle means tactile recordings have to enter the pot. Simulation will enter too, and all of the ingredients will matter as they converge.