What Is Recursive Self-improvement?

Ronak Malde says research is heading toward recursive self-improvement and continual learning, two close ways for an agent to learn from experience. Recursive self-improvement puts a model in a black box and tries to make it the best AI researcher, so the model improves itself.

That version is, in his words, a scary world, because it leaves open whether the agents will care about human interests. Continual learning still learns from experience, but from working with humans on the tasks they want done, which keeps people in the loop. He does not want agents going rogue, and staying on that alignment path is how he would avoid it.