How Does an AI Model Produce the Next Word?

Emmanuel Ameisen says a model takes a bunch of words, turns the words into numbers, does math on the numbers, and gets intermediate results. It does more math, gets more intermediate results, and repeats the cycle many times. At the end it has a final result, which it converts back into a word. The math is the part that has grown. Learning to do the task better means updating parameters in one direction or another. The model then learns some formulas, and Emmanuel Ameisen says we have no idea what the formulas are.

Training runs over a corpus of data. The model is fed the first half of a document and asked to predict the word that comes next, then the same step runs again and again across an ungodly amount of data. A fine-tuning phase follows. Emmanuel Ameisen says maybe, instead of arbitrary data, the corpus is examples of how the model should behave, and the model is trained on the examples. He describes dialogue of a helpful assistant telling a favorite recipe or solving coding problems.

The goal of Emmanuel Ameisen's team is to study what the formulas mean and what the intermediate results are, and whether the results can be decoded so people can see what the model is doing during the intermediate steps.