Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality

02/11/2022
by   Daniel Watson, et al.
6

Diffusion models have emerged as an expressive family of generative models rivaling GANs in sample quality and autoregressive models in likelihood scores. Standard diffusion models typically require hundreds of forward passes through the model to generate a single high-fidelity sample. We introduce Differentiable Diffusion Sampler Search (DDSS): a method that optimizes fast samplers for any pre-trained diffusion model by differentiating through sample quality scores. We also present Generalized Gaussian Diffusion Models (GGDM), a family of flexible non-Markovian samplers for diffusion models. We show that optimizing the degrees of freedom of GGDM samplers by maximizing sample quality scores via gradient descent leads to improved sample quality. Our optimization procedure backpropagates through the sampling process using the reparametrization trick and gradient rematerialization. DDSS achieves strong results on unconditional image generation across various datasets (e.g., FID scores on LSUN church 128x128 of 11.6 with only 10 inference steps, and 4.82 with 20 steps, compared to 51.1 and 14.9 with strongest DDPM/DDIM baselines). Our method is compatible with any pre-trained diffusion model without fine-tuning or re-training required.

READ FULL TEXT

page 1

page 9

page 13

page 16

page 17

research
04/29/2022

Fast Sampling of Diffusion Models with Exponential Integrator

The past few years have witnessed the great success of Diffusion models ...
research
06/07/2021

Learning to Efficiently Sample from Diffusion Probabilistic Models

Denoising Diffusion Probabilistic Models (DDPMs) have emerged as a power...
research
10/06/2022

On Distillation of Guided Diffusion Models

Classifier-free guided diffusion models have recently been shown to be h...
research
02/14/2023

Where to Diffuse, How to Diffuse, and How to Get Back: Automated Learning for Multivariate Diffusions

Diffusion-based generative models (DBGMs) perturb data to a target noise...
research
05/29/2023

Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models

Due to the ease of training, ability to scale, and high sample quality, ...
research
08/23/2023

Boosting Diffusion Models with an Adaptive Momentum Sampler

Diffusion probabilistic models (DPMs) have been shown to generate high-q...
research
09/10/2023

Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood

Training energy-based models (EBMs) with maximum likelihood estimation o...

Please sign up or login with your details

Forgot password? Click here to reset