Can Diffusion Models Match Cerebras Speed on NVIDIA GPUs?

Stefano Ermon says a diffusion language model can match the speed of an autoregressive model on custom hardware while running on NVIDIA GPUs. OpenCall, a company building voice agents, is the example he names.

The voice pipeline uses an ASR model, then a reasoning LLM that makes the tool calls and decides what to say next, then a text-to-speech component. The reasoning LLM is what delivers the highest quality, and speed matters for the experience. OpenCall had been serving those LLMs on Cerebras, using custom chips to reach that speed. A diffusion-based LLM built to run in parallel accelerates at the software level, so the same speed is available on NVIDIA GPUs.

GPUs are scarce, but there are more GPUs than custom chips. The switch brings more availability, lower costs, and higher quality.