How Do Language Models Rhyme Without Saying the Plan?

Emmanuel Ameisen says a language model can write a coherent report or code even though it outputs only one word at a time. Nobody writing reports or emails thinks only about the next word with an empty mind about anything that comes next. The intermediate results made while predicting a word, such as the fifth word, can be accessed and reused by every future word.

Ameisen says that reuse is how models formulate a plan. Poetry is the example, because it is hard to rhyme without planning. A line that ends on an it word has to be followed by another it word, and the word chosen can be rabbits. When models write poetry, they write a line, then the intermediate results mark an it rhyme and a plan to write more with rabbit. Every future word looks back to the plan. The model follows the plan over many words while still outputting one word at a time, and it never has to say the plan out loud. The plan stays in the intermediate results inside the model.