Learning Generative Models using Denoising Density Estimators

by   Siavash A. Bigdeli, et al.

Learning generative probabilistic models that can estimate the continuous density given a set of samples, and that can sample from that density, is one of the fundamental challenges in unsupervised machine learning. In this paper we introduce a new approach to obtain such models based on what we call denoising density estimators (DDEs). A DDE is a scalar function, parameterized by a neural network, that is efficiently trained to represent a kernel density estimator of the data. Leveraging DDEs, our main contribution is to develop a novel approach to obtain generative models that sample from given densities. We prove that our algorithms to obtain both DDEs and generative models are guaranteed to converge to the correct solutions. Advantages of our approach include that we do not require specific network architectures like in normalizing flows, ordinary differential equation solvers as in continuous normalizing flows, nor do we require adversarial training as in generative adversarial networks (GANs). Finally, we provide experimental results that demonstrate practical applications of our technique.


page 5

page 6

page 11


FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models

A promising class of generative models maps points from a simple distrib...

Density Sketches for Sampling and Estimation

We introduce Density sketches (DS): a succinct online summary of the dat...

Stein Bridging: Enabling Mutual Reinforcement between Explicit and Implicit Generative Models

Deep generative models are generally categorized into explicit models an...

Moser Flow: Divergence-based Generative Modeling on Manifolds

We are interested in learning generative models for complex geometries d...

Can denoising diffusion probabilistic models generate realistic astrophysical fields?

Score-based generative models have emerged as alternatives to generative...

Deep discriminative to kernel generative modeling

The fight between discriminative versus generative goes deep, in both th...

Smooth densities and generative modeling with unsupervised random forests

Density estimation is a fundamental problem in statistics, and any attem...

Code Repositories


Tutorial on density estimation using DDEs

view repo


Learning Generative Models using Denoising Density Estimators

view repo


Tutorial on Energy-based Generative Models using DDEs

view repo

Please sign up or login with your details

Forgot password? Click here to reset