Why Is a Puzzled Reaction a Good Sign for a Paper?

Alex Zhang says a puzzled reaction is a good sign for a paper, because it means people are not thinking about what the work is for. Someone may work on a harness because a clear goal is already shaped around the models that exist today. The recursive language model paper is a very simple idea. When it came out, people asked what the purpose was, called it too cool, and said it was just sub-agents.

He treats that response as a sign the point was missed. SWE-bench had the same shape. Ofir tells it as a story: nobody cared, and people called the task impossible, until Devin came out and the field decided it was something to hill-climb. A simple paper can look pointless and still name a goal the models of the moment do not make obvious.