Why Can't Reinforcement Learning Wait on a Superconductor?
Liam Fedus says reinforcement learning cannot wait on a superconductor, because an update that sits for a couple of days to see whether a run found a room-temperature superconductor is completely infeasible. The materials loop has three jobs. First, decide what to make: the atoms have to stay together, and the arrangement has to show the property you want, not merely be a new configuration. Next is synthesis. Even when a structure should hold, the processing conditions that bring it into being are highly non-trivial. Then characterization, because the sample does not come off the furnace labeled.
A reward that waits out the furnace is too slow and too noisy. There are not enough agents to cut the variance, and the instruments set a physical limit on how fast a run can finish. The environments that work reward a narrower task, such as naming the crystal phases actually present in an X-ray pattern, and they penalize phases that are not chemically plausible. Stitched together, those smaller environments are the discovery loop.
