Accelerating Diffusion Models via Early Stop of the Diffusion Process

05/25/2022
by   Zhaoyang Lyu, et al.
21

Denoising Diffusion Probabilistic Models (DDPMs) have achieved impressive performance on various generation tasks. By modeling the reverse process of gradually diffusing the data distribution into a Gaussian distribution, generating a sample in DDPMs can be regarded as iteratively denoising a randomly sampled Gaussian noise. However, in practice DDPMs often need hundreds even thousands of denoising steps to obtain a high-quality sample from the Gaussian noise, leading to extremely low inference efficiency. In this work, we propose a principled acceleration strategy, referred to as Early-Stopped DDPM (ES-DDPM), for DDPMs. The key idea is to stop the diffusion process early where only the few initial diffusing steps are considered and the reverse denoising process starts from a non-Gaussian distribution. By further adopting a powerful pre-trained generative model, such as GAN and VAE, in ES-DDPM, sampling from the target non-Gaussian distribution can be efficiently achieved by diffusing samples obtained from the pre-trained generative model. In this way, the number of required denoising steps is significantly reduced. In the meantime, the sample quality of ES-DDPM also improves substantially, outperforming both the vanilla DDPM and the adopted pre-trained generative model. On extensive experiments across CIFAR-10, CelebA, ImageNet, LSUN-Bedroom and LSUN-Cat, ES-DDPM obtains promising acceleration effect and performance improvement over representative baseline methods. Moreover, ES-DDPM also demonstrates several attractive properties, including being orthogonal to existing acceleration methods, as well as simultaneously enabling both global semantic and local pixel-level control in image generation.

READ FULL TEXT

page 2

page 9

page 16

page 17

research
10/27/2022

Accelerating Diffusion Models via Pre-segmentation Diffusion Sampling for Medical Image Segmentation

Based on the Denoising Diffusion Probabilistic Model (DDPM), medical ima...
research
07/21/2023

PartDiff: Image Super-resolution with Partial Diffusion Models

Denoising diffusion probabilistic models (DDPMs) have achieved impressiv...
research
09/10/2022

Diffusion Models in Vision: A Survey

Denoising diffusion models represent a recent emerging topic in computer...
research
09/15/2023

Probabilistic Constellation Shaping With Denoising Diffusion Probabilistic Models: A Novel Approach

With the incredible results achieved from generative pre-trained transfo...
research
09/15/2023

Denoising Diffusion Probabilistic Models for Hardware-Impaired Communications

Generative AI has received significant attention among a spectrum of div...
research
08/23/2023

Boosting Diffusion Models with an Adaptive Momentum Sampler

Diffusion probabilistic models (DPMs) have been shown to generate high-q...
research
03/12/2015

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

A central problem in machine learning involves modeling complex data-set...

Please sign up or login with your details

Forgot password? Click here to reset