Introduction

This article is devoted almost entirely to derivations. For a conceptual introduction to DDPM itself, see the previous post. Looking back, I am not sure I understood every part perfectly when I wrote that one either. The DDPM paper is here. It is hard to believe this difficult paper is already three years old.

Forward and Reverse Processes

Repeatedly add a tiny Gaussian perturbation with small variance $\beta$ to image $x_0$ for $T$ steps. The final output $x_T$ approaches a Gaussian distribution with the same spatial dimensions as $x_0$. The paper schedules the variance at each step and treats those values as fixed.

Forward Process $q(x_t \vert x_{t-1})$

$$q(x_t \vert x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t}x_{t-1}, \beta_t I).$$

Direct stochastic sampling is not differentiable with respect to the distribution parameters, so training uses the reparameterization trick. At every step, the previous output is scaled by $\sqrt{1-\beta_t}$ before adding noise. The two coefficients preserve unit variance when the signal and noise each have unit variance:

$$\left(\sqrt{1-\beta_t}\right)^2+\left(\sqrt{\beta_t}\right)^2 = 1.$$

Define $\alpha_t=1-\beta_t$. For $\epsilon\sim\mathcal{N}(0,I)$, one step is

$$x_t=\sqrt{\alpha_t}x_{t-1}+\sqrt{1-\alpha_t}\epsilon.$$

Expanding one more step for $t>1$ gives

$$x_t = \sqrt{\alpha_t}\sqrt{\alpha_{t-1}}x_{t-2} + \sqrt{\alpha_t}\sqrt{1-\alpha_{t-1}}\epsilon_{t-2} + \sqrt{1-\alpha_t}\epsilon_{t-1}.$$

The independent Gaussian noise terms combine into another Gaussian whose variance is the sum of their variances:

$$\sqrt{\alpha_t}\sqrt{1-\alpha_{t-1}}\epsilon_{t-2} +\sqrt{1-\alpha_t}\epsilon_{t-1} \sim \sqrt{1-\alpha_t\alpha_{t-1}}\epsilon.$$

Repeating this argument and defining $\bar{\alpha}_t=\prod_{s=1}^t\alpha_s$ yields

$$q(x_t \vert x_0) = \mathcal{N}(x_t; \sqrt{\bar{\alpha}_t}x_0, (1-\bar{\alpha}_t)I).$$

Reparameterizing this marginal gives the formula used throughout the loss derivation:

$$x_t := \sqrt{\bar{\alpha}_t} x_0 + \sqrt{1-\bar{\alpha}_t} \epsilon,\qquad \epsilon \sim \mathcal{N}(0, I).$$

Reverse Process $p_\theta(x_{t-1} \vert x_t)$

The learned reverse process imitates the posterior of the forward process. This is distribution prediction. Wanting the reverse direction to match the forward one should sound familiar: the approach resembles a VAE and therefore optimizes a variational lower bound.

The reverse joint from $x_T$ to $x_0$ factors as

$$p_\theta(x_{0:T}) := p(x_T) \prod_{t=1}^T p_\theta(x_{t-1} \vert x_t),$$

with each transition modeled as

$$p_\theta(x_{t-1} \vert x_t) := \mathcal{N}(x_{t-1}; \mu_{\theta}(x_t,~t), \Sigma_\theta (x_t,~t)).$$

The forward process adds a very small Gaussian at each step, so the reverse removes a very small Gaussian. The network can sample backward by predicting the noise at each point and subtracting it.

The loss should therefore make the learned reverse process match the known forward process.

Loss Function

Evidence Lower Bound in DDPM

A VAE negative-log-likelihood bound can be written

$$\mathbb{E}[-\log p_\theta (x_0)] \leq \mathbb{E}_q \left[ -\log p_\theta(x_0 \vert z) -\log \frac{p_\theta(z)}{q_\phi(z \vert x_0)} \right].$$

The first term trains the decoder to reconstruct $x_0$ from $z$. The second is a KL-divergence term that makes encoder posterior $q_\phi(z\mid x_0)$ resemble a predefined latent prior such as Gaussian $p_\theta(z)$.

Rearranging the same log ratio to emphasize the generative joint,

$$\mathbb{E}[-\log p_\theta (x_0)] \leq \mathbb{E}_q \left[ -\log p_\theta(z) -\log \frac{p_\theta(x_0 \vert z)}{q_\phi(z \vert x_0)} \right].$$

Replace latent $z$ with a sequence of time-indexed variables and remove a learned encoder—DDPM's forward process $q$ is fixed—to obtain

$$\mathbb{E}[-\log p_\theta (x_0)] \leq \mathbb{E}_q \left[ -\log p_\theta(x_T) -\log \frac{p_\theta(x_{0:T-1} \vert x_T)}{q(x_{1:T} \vert x_0)} \right].$$

Both joints are Markov chains and factor over steps:

$$\mathbb{E}[-\log p_\theta (x_0)] \leq \mathbb{E}_q \left[ -\log p_\theta(x_T) -\log \prod_{t \ge 1} \frac{p_\theta(x_{t-1} \vert x_t)}{q(x_t \vert x_{t-1})} \right].$$

Moving the product outside the logarithm turns it into a sum:

$$\mathbb{E}[-\log p_\theta (x_0)] \leq \mathbb{E}_q \left[ -\log p_\theta(x_T) - \sum_{t \ge 1}\log \frac{p_\theta(x_{t-1} \vert x_t)}{q(x_t \vert x_{t-1})} \right].$$

Why arrange the VAE terms differently? Intuitively, we want the forward and reverse processes to match. More precisely, a VAE decoder learns an implicit conditional distribution $p_\theta(x_0\mid z)$, whereas a diffusion decoder learns the stepwise dynamics that reverse the diffusion process.

The problem is that although the forward transition $q(x_t\mid x_{t-1})$ is known, its reverse posterior $q(x_{t-1}\mid x_t)$ is intractable:

$$q(x_{t-1} \vert x_t) = \frac{q(x_t \vert x_{t-1})q(x_{t-1})}{q(x_t)}.$$

An intuitive explanation: because we know the noise added to $x_{t-1}$, the conditional distribution of $x_t$ given $x_{t-1}$ is explicitly Gaussian. But we do not know the marginal distributions of noisy states $q(x_t)$ and $q(x_{t-1})$, leaving two intractable terms in Bayes' rule.

Conditioning every term on the original sample $x_0$ makes the posterior tractable because accumulated Gaussian noise from $x_0$ has the closed form derived above:

$$\begin{aligned} q(x_{t-1} \vert x_t,x_0) &= \frac{q(x_t \vert x_{t-1},x_0)q(x_{t-1},x_0)}{q(x_t,x_0)} \newline &= \frac{q(x_t \vert x_{t-1},x_0)q(x_{t-1}\vert x_0)q(x_0)}{q(x_t \vert x_0)q(x_0)} \newline &= q(x_t \vert x_{t-1},x_0) \frac{q(x_{t-1} \vert x_0)}{q(x_t \vert x_0)}. \end{aligned}$$

Adding $x_0$ may appear to make this a different target from $p_\theta(x_{t-1}\mid x_t)$. During training, however, $x_0$ is known and the Markov property provides the tractable posterior whose conditional mean the model can learn from $x_t$.

The decomposed negative ELBO is

$$D_{KL}(q(x_T \vert x_0) \Vert p_\theta(x_T)) +\sum_{t > 1} D_{KL} (q(x_{t-1} \vert x_t, x_0) \Vert p_\theta(x_{t-1} \vert x_t)) -\mathbb{E}_q[\log p_{\theta}(x_0 \vert x_1)].$$

Starting from the factorization and replacing the $t>1$ terms with the tractable posterior gives

$$\begin{aligned} \mathcal{L} \le& \mathbb{E}_q\left[-\log p_\theta(x_T) -\sum_{t=2}^T \log \frac{p_\theta(x_{t-1} \vert x_t)}{q(x_t \vert x_{t-1})} -\log \frac{p_\theta(x_0 \vert x_1)}{q(x_1 \vert x_0)} \right] \newline =& \mathbb{E}_q\left[-\log p_\theta(x_T) -\sum_{t=2}^T \log \left( \frac{p_\theta(x_{t-1} \vert x_t)}{q(x_{t-1} \vert x_t, x_0)} \frac{q(x_{t-1} \vert x_0)}{q(x_t\vert x_0)}\right) -\log \frac{p_\theta(x_0 \vert x_1)}{q(x_1 \vert x_0)} \right] \newline =& \sum_{t > 1} D_{KL} (q(x_{t-1} \vert x_t, x_0) \Vert p_\theta(x_{t-1} \vert x_t)) \ &+\mathbb{E}_q\left[-\log p_\theta(x_T) -\log \frac{q(x_1 \vert x_0)}{q(x_T \vert x_0)} -\log \frac{p_\theta(x_0 \vert x_1)}{q(x_1 \vert x_0)} \right] \newline =& D_{KL}(q(x_T \vert x_0) \Vert p_\theta(x_T)) +\sum_{t > 1} D_{KL} (q(x_{t-1} \vert x_t, x_0) \Vert p_\theta(x_{t-1} \vert x_t)) \ &-\mathbb{E}_q[\log p_{\theta}(x_0 \vert x_1)]. \end{aligned}$$

From ELBO to an Objective

The expression above is still abstract. The implemented objective follows after substituting the Gaussian distributions. The ELBO has three pieces.

The first KL term is naturally satisfied by the forward diffusion schedule, which drives $x_T$ toward the chosen Gaussian prior, and it contains no learned reverse parameters in the usual setup:

$$D_{KL}(q(x_T \vert x_0) \Vert p_\theta(x_T)).$$

In a VAE, a similar term regularizes a learned encoder. Diffusion does not learn $q$, so this term is normally omitted from the trainable loss.

The final log-likelihood term is less intuitive:

$$\mathbb{E}_q[\log p_{\theta}(x_0 \vert x_1)].$$

It assigns likelihood to reconstructing clean $x_0$ from slightly noised $x_1$. The middle terms compare Gaussian distributions via KL divergence and do not require mapping the density into a discrete pixel probability. This endpoint term does.

Consider only

$$p_\theta(x_0 \vert x_1).$$

The reverse network predicts a Gaussian distribution over every component of $x_0$:

$$p_\theta(x_0 \vert x_1) \sim \mathcal{N}(x_0;~\mu_\theta(x_1, 1), \sigma_1^2).$$

To obtain the likelihood of a discrete image, integrate this Gaussian over the quantization bin for every pixel. Assume images are normalized to $[-1,1]$. Values below $-1$ map to the boundary pixel $-1$, values above $1$ map to $1$, and intermediate 8-bit values occupy bins of width approximately $2/255$.

Suppose the ground-truth component should be zero when mapping $x_1$ back to $x_0$.

Then $p_\theta(x_0\mid x_1)$ assigns the area of the predicted density around that quantized value—the yellow region below.

Multiplying over $D$ image components gives the joint probability:

$$p_\theta(x_0 \vert x_1) = \prod_{i=1}^D \int_{\delta_{-}(x_0^i)}^{\delta_+(x_0^i)} \mathcal{N}(x; \mu_\theta^i (x_1, 1), \sigma_1^2)\,dx.$$

The bin boundaries clamp the endpoints:

$$\delta_+(x) = \begin{cases} \infty,&\text{if }x=1 \newline x+\frac{1}{255},&\text{if }x<1 \end{cases},\qquad \delta_-(x) = \begin{cases} -\infty,&\text{if }x=-1 \newline x-\frac{1}{255},&\text{if }x>-1. \end{cases}$$

Optimizing the Intermediate Steps $L_{1:T-1}$

The discussion above says that $L_T$ requires no learned optimization and that $L_0$ can be evaluated through discretized Gaussian likelihood. What about the intermediate denoising terms?

$$\sum_{t > 1} D_{KL} (q(x_{t-1} \vert x_t, x_0) \Vert p_\theta(x_{t-1} \vert x_t)).$$

We first show that $q(x_{t-1}\mid x_t,x_0)$ is Gaussian. The tractable posterior is

$$q(x_{t-1} \vert x_t,x_0) = q(x_t \vert x_{t-1},x_0) \frac{q(x_{t-1} \vert x_0)}{q(x_t \vert x_0)}.$$

Every factor is known in Gaussian form:

$$\begin{aligned} q(x_t \vert x_0) &\sim \mathcal{N}(\sqrt{\bar{\alpha}_t}x_0, (1-\bar{\alpha}_t)I) \newline q(x_t \vert x_{t-1}) &\sim \mathcal{N}(\sqrt{\alpha_t}x_{t-1},\beta_tI). \end{aligned}$$

Substitute their Gaussian densities and ignore normalization constants that do not depend on $x_{t-1}$:

$$\begin{aligned} q(x_{t-1} \vert x_t, x_0) \propto& \exp \left[-\frac{1}{2} \left( \frac{(x_t - \sqrt{\alpha_t}x_{t-1})^2}{\beta_t} + \frac{(x_{t-1} - \sqrt{\bar{\alpha}_{t-1}}x_0)^2}{1-\bar{\alpha}_{t-1}} - \frac{(x_t - \sqrt{\bar{\alpha}_t}x_0)^2}{1-\bar{\alpha}_t} \right) \right] \newline =& \exp \left[-\frac{1}{2} \left( \frac{x_t^2 - 2\sqrt{\alpha_t}x_t x_{t-1}+\alpha_tx_{t-1}^2}{\beta_t} + \frac{x_{t-1}^2 - 2\sqrt{\bar{\alpha}_{t-1}}x_0x_{t-1} + \bar{\alpha}_{t-1}x_0^2}{1-\bar{\alpha}_{t-1}} - \frac{x_t^2 -2\sqrt{\bar{\alpha}_t}x_0x_t + \bar{\alpha}_tx_0^2}{1-\bar{\alpha}_t} \right) \right]. \end{aligned}$$

Collect terms in $x_{t-1}$:

$$\exp \left[-\frac{1}{2} \left( \left( \frac{\alpha_t}{\beta_t} +\frac{1}{1-\bar{\alpha}_{t-1}} \right)x_{t-1}^2 - \left(2\frac{\sqrt{\alpha_t}x_t}{\beta_t} + 2\frac{\sqrt{\bar{\alpha}_{t-1}}x_0}{1-\bar{\alpha}_{t-1}}\right)x_{t-1} + C(x_t, x_0) \right) \right].$$

Completing the square—or using the vertex $-b/(2a)$—gives posterior mean

$$\begin{aligned} \tilde{\mu}_t(x_t,~x_0) &= \frac{\frac{\sqrt{\alpha_t}x_t}{\beta_t} + \frac{\sqrt{\bar{\alpha}_{t-1}}x_0}{1-\bar{\alpha}_{t-1}}} {\frac{\alpha_t}{\beta_t} +\frac{1}{1-\bar{\alpha}_{t-1}}} \newline &= \frac{\sqrt{\alpha_t}(1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_t}x_t + \frac{\beta_t \sqrt{\bar{\alpha}_{t-1}}}{1-\bar{\alpha}_t}x_0, \end{aligned}$$

and posterior variance

$$\tilde{\beta}_t = \tilde{\sigma}_t^2 = \beta_t \left( \frac{1-\bar{\alpha}_{t-1}}{1-\bar{\alpha}_t} \right).$$

The DDPM paper moves past this algebra quickly, which is why I wanted to leave the full derivation here.

The model distribution is

$$p_\theta(x_{t-1}\mid x_t) =\mathcal{N}(x_{t-1};\mu_\theta(x_t,t),\Sigma_\theta(x_t,t)).$$

In the original simplified setup, its variance is fixed rather than learned, often to $\beta_t$ or $\tilde{\beta}_t$. The experiments report little difference between those choices. The network therefore needs to predict only $\mu_\theta(x_t,t)$.

With equal fixed covariance, KL divergence between the two Gaussians reduces, up to parameter-independent constant $C$, to squared error between their means:

$$L_{t-1} := \mathbb{E}_q \left[ \frac{1}{2\sigma_t^2} \left\lVert \tilde{\mu}_t(x_t, x_0) - \mu_\theta(x_t, t)\right\rVert^2 \right]+C.$$

To express the stochastic $x_t$ differentiably, substitute

$$x_t(x_0,\epsilon) =\sqrt{\bar{\alpha}_t}x_0+\sqrt{1-\bar{\alpha}_t}\epsilon.$$

The original sample can likewise be written from $x_t$ and noise:

$$x_0=\frac{x_t}{\sqrt{\bar{\alpha}_t}} -\frac{\sqrt{1-\bar{\alpha}_t}}{\sqrt{\bar{\alpha}_t}}\epsilon.$$

Then

$$\begin{aligned} L_{t-1}-C = \mathbb{E}_q \Bigg[ \frac{1}{2\sigma_t^2}\Bigg\lVert \tilde{\mu}_t\left( x_t(x_0,\epsilon), \frac{x_t(x_0,\epsilon)}{\sqrt{\bar{\alpha}_t}} -\frac{\sqrt{1-\bar{\alpha}_t}}{\sqrt{\bar{\alpha}_t}}\epsilon \right) -\mu_\theta(x_t(x_0,\epsilon),t) \Bigg\rVert^2\Bigg]. \end{aligned}$$

Substitute into the posterior mean:

$$\begin{aligned} \tilde{\mu}_t(x_t,x_0) &= \frac{\sqrt{\alpha_t}(1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_t}x_t + \frac{\beta_t\sqrt{\bar{\alpha}_{t-1}}}{1-\bar{\alpha}_t}x_0 \newline &= \frac{1}{\sqrt{\alpha_t}}x_t(x_0,\epsilon) -\frac{\beta_t}{\sqrt{(1-\bar{\alpha}_t)\alpha_t}}\epsilon \newline &= \frac{1}{\sqrt{\alpha_t}} \left(x_t(x_0,\epsilon)-\frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}}\epsilon\right). \end{aligned}$$

The mean-matching objective is therefore

$$\mathbb{E}_q \left[ \frac{1}{2\sigma_t^2} \left\lVert \frac{1}{\sqrt{\alpha_t}} \left(x_t(x_0,\epsilon)-\frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}}\epsilon\right) -\mu_\theta(x_t(x_0,\epsilon),t) \right\rVert^2 \right].$$

At time $t$, $x_t$ is already the network input. The unknown part of the forward construction is the added noise $\epsilon$, so parameterize the model mean through a noise predictor $\epsilon_\theta$:

$$\mu_\theta(x_t, t) = \frac{1}{\sqrt{\alpha_t}} \left(x_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}} \epsilon_\theta(x_t, t)\right).$$

Sampling $x_{t-1}$ uses this predicted mean plus the fixed variance term:

$$x_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left(x_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}} \epsilon_\theta(x_t, t)\right)+\sigma_t z,\qquad z\sim\mathcal{N}(0,I).$$

Finally, substituting the mean parameterization reduces the intermediate objective to weighted noise-prediction error:

$$\mathbb{E}_{x_0,\epsilon} \left[ \frac{\beta_t^2}{2\sigma_t^2 \alpha_t(1-\bar{\alpha}_t)} \left\lVert \epsilon - \epsilon_\theta(\sqrt{\bar{\alpha}_t}x_0 + \sqrt{1-\bar{\alpha}_t}\epsilon, t) \right\rVert^2\right].$$

This is the derivation behind the familiar DDPM noise-prediction objective.