Diffusion Models Explained: The DDPM Paper Behind Every AI Image Generator

"Denoising Diffusion Probabilistic Models" by Jonathan Ho, Ajay Jain and Pieter Abbeel of UC Berkeley (arXiv 2006.11239, submitted 19 June 2020) trained a network to predict noise and reported an FID of 3.17 and an Inception score of 9.46 on unconditional CIFAR-10. Those are the authors' numbers, and they beat the unconditional GAN and score-based baselines in the paper's own table. The same paper also admits its log-likelihoods were not competitive, which is the part most summaries skip.

What problem does the DDPM paper solve?

It makes an old, hard-to-train idea work: generate images by learning to reverse a gradual noising process. The idea was proposed in 2015 by Sohl-Dickstein, Weiss, Maheswaranathan and Ganguli, who suggested to "systematically and slowly destroy structure in a data distribution" and then learn the reverse. In the DDPM paper's CIFAR-10 table, that original diffusion model is listed at a negative log-likelihood of at most 5.40 bits/dim and has no reported sample-quality scores.

The other thread was score-based modelling. Song and Ermon (arXiv 1907.05600, July 2019) sampled with annealed Langevin dynamics from learned noise-conditional scores. DDPM's table lists their model (NCSN) at IS 8.87 and FID 25.32.

DDPM's contribution was a training recipe and network that moved the FID from 25.32 to 3.17 on the same benchmark, and a clear link between the two threads.

How do diffusion models work in the DDPM paper?

A fixed forward process corrupts an image into noise over T = 1000 steps, and a network learns to undo one step at a time. The noise variances increase linearly from 10^-4 to 0.02, per the full text.

With alpha_t = 1 - beta_t and abar_t as the running product of the alphas, the noisy image at any step has a closed form, so training never simulates the chain:

x_t = sqrt(abar_t) * x_0 + sqrt(1 - abar_t) * eps,   eps ~ N(0, I)

The training loop is short:

repeat:
  x_0  ~ data
  t    ~ Uniform{1..T}
  eps  ~ N(0, I)
  step on || eps - eps_theta( sqrt(abar_t) x_0 + sqrt(1-abar_t) eps , t ) ||^2

That squared error is the paper's L_simple. It is a reweighted version of the variational bound that drops a per-step weight, which down-weights small t so the network can focus on harder denoising at large t. Sampling starts from Gaussian noise and applies the learned reverse step 1,000 times, adding fresh noise at each step except the last. The authors note this resembles Langevin dynamics with the network acting as a learned gradient of the data density, and that the objective resembles denoising score matching over multiple noise scales. They call that equivalence one of their primary contributions.

The network is a U-Net built on a Wide ResNet, with group normalisation, a sinusoidal embedding of t added into each residual block, and self-attention at the 16x16 feature-map resolution. The CIFAR-10 model has 35.7M parameters. The 256x256 models have 114M, and the large LSUN Bedroom model has about 256M.

What are the results?

On CIFAR-10 the simplified objective gave an FID of 3.17 and IS of 9.46, lab-reported, with 50,000 samples. The table below shows unconditional CIFAR-10 rows from the paper's Table 1.

The ablation shows what mattered. Predicting the noise with the full bound and fixed variance gave FID 13.51; switching to the simplified loss gave 3.17. In the paper's ablation, learned reverse variances with epsilon prediction were unstable to train and produced poor samples.

On 256x256 LSUN, DDPM was good but not best. FIDs for L_simple were 6.36 on Bedroom (4.90 for the larger model), 7.89 on Church and 19.75 on Cat. StyleGAN scored 2.65 on Bedroom, and StyleGAN2 scored 3.86 on Church and 6.93 on Cat.

Does the evidence hold up?

Mostly for sample quality on CIFAR-10, with four caveats a researcher should keep in view.

1. One small benchmark. The headline is a single model on 32x32 images. The paper shows LSUN and CelebA-HQ samples at 256x256, but gives FIDs only for LSUN, and there StyleGAN variants lead on all three categories.
2. The reference set matters. The 3.17 FID is computed against the training set. Against the test set it is 5.24. Comparisons across papers need the same convention.
3. Likelihood is weak. At up to 3.75 bits/dim, DDPM trails Gated PixelCNN (3.03) and Sparse Transformer (2.80). The authors write that their models "do not have competitive log likelihoods" and that most of the lossless codelength goes to imperceptible detail. In their rate-distortion split, the best model spends 1.78 bits/dim on rate and 1.97 on distortion.
4. Not overfitting, as far as bits/dim shows. The train-test gap is at most 0.03 bits/dim, so the sample quality is not simply memorisation by that measure. The paper does not claim to rule out memorisation by other means.

Code is public at hojonathanho/diffusion. All numbers above are the authors' claims from the paper.

How does DDPM compare with related work?

Four follow-ups explain why the paper became a foundation. Figures below are the authors' abstract-level claims.

The Dhariwal and Nichol abstract also says their model matches BigGAN-deep with as few as 25 forward passes per sample. The latent diffusion abstract says pixel-space models often consume hundreds of GPU days, which motivated moving the process into a compressed space where text or bounding boxes can condition generation through cross-attention. If that attention mechanism is new to you, our decode of Attention Is All You Need covers it.

What does a working researcher take from this?

Four practical lessons come out of the paper's own ablations and design choices.

• Predict the noise, not the image or the mean. Epsilon prediction with L_simple gave FID 3.17, against 13.51 with the same prediction target under the full bound.
• Fixed variances were the stable choice here. The authors report that learned variances were unstable to train and generated poor samples. Later work (Improved DDPM) found a way to learn them and use far fewer steps.
• Do not read FID and likelihood as the same thing. The best samples came from the objective with the worse codelength. Pick the metric that matches your use.
• Check the FID reference set. Always state whether statistics come from the train or test split.

For readers following how scale changes conclusions in other model families, the Kaplan scaling-laws decode is a useful contrast: Improved DDPM reports that sample quality and likelihood scale smoothly with model capacity and training compute, which is the same style of claim.

What's still unclear?

The 1,000-step sampler is the obvious cost. The authors set T = 1000 so the number of network evaluations "matches previous work", and report 17 seconds for 256 CIFAR-10 images and 300 seconds for 128 images at 256x256 on a TPU v3-8. Speedups from DDIM and Improved DDPM are the authors' claims on their own setups, and the abstracts give no single number that applies across models.

Three further questions remain open from this paper alone:

• Why the weaker-bound objective gives better samples is explained only informally (down-weighting easy small-t tasks).
• The compression scheme the authors sketch relies on minimal random coding, which they say is not tractable for high-dimensional data, so the lossy-compression framing is a proof of concept.
• The broader-impact section raises misuse, such as fake images, and dataset bias.

What's next?

The paper's own list is short: a more powerful decoder, such as a conditional autoregressive model, is left to future work, and the authors suggest uses in compression. The field moved in the directions the table above shows: faster samplers, learned variances, guidance, latent-space generation, and a continuous-time view (Song and colleagues, arXiv 2011.13456, November 2020). For how research labs build on a landmark result and how human-feedback tuning changed language models, see our InstructGPT decode.