Generative Modeling with Denoising Auto-Encoders and Langevin Sampling

01/31/2020
by   Adam Block, et al.
1

We study convergence of a generative modeling method that first estimates the score function of the distribution using Denoising Auto-Encoders (DAE) or Denoising Score Matching (DSM) and then employs Langevin diffusion for sampling. We show that both DAE and DSM provide estimates of the score of the Gaussian smoothed population density, allowing us to apply the machinery of Empirical Processes. We overcome the challenge of relying only on L^2 bounds on the score estimation error and provide finite-sample bounds in the Wasserstein distance between the law of the population distribution and the law of this sampling scheme. We then apply our results to the homotopy method of arXiv:1907.05600 and provide theoretical justification for its empirical success.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/27/2023

Denoising Diffusion Samplers

Denoising diffusion models are a popular class of generative models prov...
research
05/13/2021

On the Bahadur representation of sample quantiles for score functionals

We establish the Bahadur representation of sample quantiles for stabiliz...
research
09/22/2022

Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

We provide theoretical convergence guarantees for score-based generative...
research
11/07/2022

From Denoising Diffusions to Denoising Markov Models

Denoising diffusions are state-of-the-art generative models which exhibi...
research
05/29/2013

Generalized Denoising Auto-Encoders as Generative Models

Recent work has shown how denoising and contractive autoencoders implici...
research
04/08/2021

On tuning consistent annealed sampling for denoising score matching

Score-based generative models provide state-of-the-art quality for image...
research
03/03/2023

Diffusion Models are Minimax Optimal Distribution Estimators

While efficient distribution learning is no doubt behind the groundbreak...

Please sign up or login with your details

Forgot password? Click here to reset