Diffusion model
A diffusion model is a generative model that learns to create data by reversing a gradual noising process. Starting from random noise, it iteratively denoises toward a realistic sample, and it is the technique behind most state-of-the-art image and video generation.
In this article
Building visual or physical AI?
Let's talk.
What is a diffusion model?
A diffusion model generates data by learning to undo noise. During training it takes real samples, adds noise in many small steps until they become pure static, and learns to predict and remove that noise. To generate, it starts from random noise and applies the learned denoising steps repeatedly until a coherent sample emerges.
This step-by-step approach produces high-quality, diverse outputs and has become the dominant method for image, video, and increasingly other kinds of generation.
Key takeaways
- Diffusion models generate by reversing a gradual noising process.
- They denoise from random noise to a realistic sample over many steps.
- They lead current image and video generation in quality and diversity.
What generative approaches trade off
| Approach | Characteristic |
|---|---|
| Diffusion model | High quality and diversity, slower multi-step sampling |
| Generative adversarial network | Fast sampling, can be unstable to train |
| Autoencoder | Compact representations, typically blurrier samples |
How it works
A forward process defines how noise is added over a schedule of steps. A neural network, often a U-Net or transformer, is trained to estimate the noise at each step so it can be removed. Generation runs the reverse process from noise to data, and conditioning inputs such as a text prompt steer what gets generated.
Why it matters
Diffusion models set the standard for controllable, high-fidelity generation, which matters for creating synthetic training data, augmenting rare cases, and simulating scenarios. Their ability to condition on text and other signals also ties them closely to multimodal systems.
Frequently asked questions
How is a diffusion model different from a GAN?
A GAN generates in a single pass through a generator trained against a discriminator, while a diffusion model generates through many denoising steps, usually with higher quality but slower sampling.
Why does generation take multiple steps?
Each step removes a little noise, and repeating the learned denoising many times is what turns random noise into a coherent sample.
Related terms
Director of Growth
Jesse Mostipak is the Director of Growth at Voxel51, where the work is helping humans find and trust what the brand knows, and teaching the Google knowledge graph and the LLMs answering on their behalf to do the same. That question, how knowledge gets built inside a system, is one Jesse has been chasing for years. Earlier versions of it ran through a New York City high school science classroom, data science and machine learning, and developer relations at Kaggle, Posit (formerly RStudio), and Baseten. The answer doesn't change much depending on whether the learner is a teenager, a software engineer, or a knowledge graph. Jesse holds a Master's in Education from CUNY Hunter College. LinkedIn
See all articles by Jesse Mostipak
Last updated November 3, 2026
Building visual or physical AI?
Let's talk.
© 2026 Voxel51 All Rights Reserved