A diffusion model is a generative model that learns to create data by reversing a gradual noising process. Starting from random noise, it iteratively denoises toward a realistic sample, and it is the technique behind most state-of-the-art image and video generation.
A diffusion model generates data by learning to undo noise. During training it takes real samples, adds noise in many small steps until they become pure static, and learns to predict and remove that noise. To generate, it starts from random noise and applies the learned denoising steps repeatedly until a coherent sample emerges.
This step-by-step approach produces high-quality, diverse outputs and has become the dominant method for image, video, and increasingly other kinds of generation.
Key takeaways
Diffusion models generate by reversing a gradual noising process.
They denoise from random noise to a realistic sample over many steps.
They lead current image and video generation in quality and diversity.
What generative approaches trade off
How diffusion compares to other generative model families.
How diffusion compares to other generative model families.
Approach
Characteristic
Diffusion model
High quality and diversity, slower multi-step sampling
Generative adversarial network
Fast sampling, can be unstable to train
Autoencoder
Compact representations, typically blurrier samples
How it works
A forward process defines how noise is added over a schedule of steps. A neural network, often a U-Net or transformer, is trained to estimate the noise at each step so it can be removed. Generation runs the reverse process from noise to data, and conditioning inputs such as a text prompt steer what gets generated.
Why it matters
Diffusion models set the standard for controllable, high-fidelity generation, which matters for creating synthetic training data, augmenting rare cases, and simulating scenarios. Their ability to condition on text and other signals also ties them closely to multimodal systems.
Frequently asked questions
How is a diffusion model different from a GAN?
A GAN generates in a single pass through a generator trained against a discriminator, while a diffusion model generates through many denoising steps, usually with higher quality but slower sampling.
Why does generation take multiple steps?
Each step removes a little noise, and repeating the learned denoising many times is what turns random noise into a coherent sample.