How Did Periodic Labs Catch a Bad Machine Load?
Liam Fedus says Periodic Labs caught a bad machine load because an AI system with a full view of the lab data noticed a pattern no single run made obvious. One step had a cyclic error. The patterns were inconsistent with what had been run. The system read through the runs, judged the result unexpected, and looked at the basket of data over time. Reversing a cyclic permutation made the runs consistent. The team went back to the physical equipment and confirmed that the machine had been loaded incorrectly.
He wants that lab to produce a huge quantity of data, and for the data to be high quality and diverse. Robots may make fewer mistakes than people on the routine steps. Managing data quality is foundational to doing AI in the physical world, because a model trained on a scrambled log does not learn the experiment that was actually run.