Smash Notes

Back to Why Diffusion Will Win AI Inference with Inception Co-Founder and CEO Stefano Ermon, with Stefano Ermon.

Why Didn't Inception Open Source Its Diffusion Models?

Stefano Ermon says Inception did not open source everything in part to keep the IP a little closer to the company. Diffusion is a different stack, and a lot of it had to be built in-house because there is no mature ecosystem for the serving engine or the kernels.

Much of the work had to be developed inside the company. Some open source models exist, but they are not particularly good, so the models are a little harder to deploy and customers are not used to trying them. Inception has to build its own stack for SFT, for RLHF, and for RL, and the stack becomes IP to some extent. One reason for keeping it closed was to hold that IP closer and not open it.

The closed choice has downsides. The community has less chance to contribute, adoption is harder, and on-prem deployments are hard. There are pros and cons with both choices.

Grokipedia·Inception

Transcript

Why Diffusion Will Win AI Inference with Inception Co-Founder and CEO Stefano Ermon ·

No Priors: Artificial Intelligence | Technology | Startups

More from this episode

  1. How Do Diffusion Models Write Text Faster Than Autoregressive Models?Stefano Ermon says a diffusion model generated text 10x faster than an autoregressive model with the same number of parameters, because it outputs many tokens…
  2. Are Inception's Mercury Models Faster Than Haiku?Stefano Ermon says Inception's Mercury models are on par with the Haiku models, the Flash models, and OpenAI's mini and nano models on benchmarks, while being…
  3. How Does a Diffusion Language Model Trade Compute for Quality?Stefano Ermon says a diffusion-based model gives Inception many ways to trade compute for quality at inference time.
  4. Can Diffusion Models Match Cerebras Speed on NVIDIA GPUs?Stefano Ermon says a diffusion language model can match the speed of an autoregressive model on custom hardware while running on NVIDIA GPUs.
  5. How Do Diffusion Models Learn Structure by Compressing Data?Stefano Ermon says training a generative model finds structure in data by building a compression scheme.
  6. Are Diffusion Models Easier to Control Than Autoregressive Models?Stefano Ermon says diffusion models are typically easier to control than autoregressive models.

People

More answers like this

More with Sarah Guo

Related episodes

Plays from this note through the rest of the episode.

Note

Weekly highlights

One letter a week.

The notes worth hearing, with the minute they were said.