Why Does Dyna 2 Use a Diffusion Transformer?

Jason Ma says the Dyna 2 world action model is a diffusion transformer. Building a robotics model comes down to the data and to how much the model can learn from it, and Dyna 2 uses the same transformer style as a language model, sequence to sequence and decoder only, with diffusion on top.

A diffusion model starts from random noise and, through repeated computation, turns the noise into something high dimensional with very fine detail. Image generators need sharp edges, sharp corners, and strong color contrast. Robot actions need the same precision and smoothness, so the field trains robotics foundation models as diffusion transformers. The transformer is what can take in the most data. Diffusion then refines the result into smooth, fine-grained motion the robot can execute.

Flow matching is a simpler special case of diffusion and mostly a speedup. A normal run takes 40 to 100 diffusion steps, and flow matching can cut that to four, with less scheduler tuning and results that work just as well. About two to three years ago, most image and video generative modeling shifted from diffusion to flow matching, while the idea stayed the same. On a robot, the short loop also lets the model re-check the action in real time, unlike an image that is generated in one shot.