What Is Alison Gopnik's Curiosity-based Learning?

On Danny in the Valley, Alison Gopnik of UC Berkeley describes work with Berkeley computer scientists, including Pulkit Agrawal and Deepak Pathak. They call it curiosity-based reinforcement learning, drawn from developmental psychology. Ordinary reinforcement learning tells the agent that a score went up or down. Here, the reward is for a prediction that does not fit what you already know. When something unexpected happens, you go back and try to make sense of it. Gopnik says that looks like what children do. Her lab was giving the same problems to four-year-olds and to the curiosity-based AI, to compare what each one did.