PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Driven Adaptive Prior

by   Sang-gil Lee, et al.

Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density. The framework assumes the prior noise as a standard Gaussian distribution, whereas the corresponding data distribution may be more complicated than the standard Gaussian distribution, which potentially introduces inefficiency in denoising the prior noise into the data sample because of the discrepancy between the data and the prior. In this paper, we propose PriorGrad to improve the efficiency of the conditional diffusion model (for example, a vocoder using a mel-spectrogram as the condition) by applying an adaptive prior derived from the data statistics based on the conditional information. We formulate the training and sampling procedures of PriorGrad and demonstrate the advantages of an adaptive prior through a theoretical analysis. Focusing on the audio domain, we consider the recently proposed diffusion-based audio generative models based on both the spectral and time domains and show that PriorGrad achieves a faster convergence leading to data and parameter efficiency and improved quality, and thereby demonstrating the efficiency of a data-driven adaptive prior.


page 2

page 3

page 6

page 7

page 9

page 10

page 11

page 15


Denoising Diffusion Gamma Models

Generative diffusion processes are an emerging and effective tool for im...

On Fast Sampling of Diffusion Probabilistic Models

In this work, we propose FastDPM, a unified framework for fast sampling ...

Diffusion models as plug-and-play priors

We consider the problem of inferring high-dimensional data 𝐱 in a model ...

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

A wide variety of deep generative models has been developed in the past ...

Score-based diffusion models for accelerated MRI

Score-based diffusion models provide a powerful way to model images usin...

Entropy-driven Sampling and Training Scheme for Conditional Diffusion Generation

Denoising Diffusion Probabilistic Model (DDPM) is able to make flexible ...

Flow-Based Likelihoods for Non-Gaussian Inference

We investigate the use of data-driven likelihoods to bypass a key assump...