DeepAI AI Chat
Log In Sign Up

Theoretical guarantees for sampling and inference in generative models with latent diffusions

by   Belinda Tzen, et al.
University of Illinois at Urbana-Champaign

We introduce and study a class of probabilistic generative models, where the latent object is a finite-dimensional diffusion process on a finite time interval and the observed variable is drawn conditionally on the terminal point of the diffusion. We make the following contributions: We provide a unified viewpoint on both sampling and variational inference in such generative models through the lens of stochastic control. We quantify the expressiveness of diffusion-based generative models. Specifically, we show that one can efficiently sample from a wide class of terminal target distributions by choosing the drift of the latent diffusion from the class of multilayer feedforward neural nets, with the accuracy of sampling measured by the Kullback-Leibler divergence to the target distribution. Finally, we present and analyze a scheme for unbiased simulation of generative models with latent diffusions and provide bounds on the variance of the resulting estimators. This scheme can be implemented as a deep generative model with a random number of layers.


page 1

page 2

page 3

page 4


Expressiveness Remarks for Denoising Diffusion Models and Samplers

Denoising diffusion models are a class of generative models which have r...

Diffusion Schrödinger Bridges for Bayesian Computation

Denoising diffusion models are a novel class of generative models that h...

Stochastic Interpolants: A Unifying Framework for Flows and Diffusions

We introduce a class of generative models based on the stochastic interp...

Improving Diffusion Model Efficiency Through Patching

Diffusion models are a powerful class of generative models that iterativ...

Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit

In deep latent Gaussian models, the latent variable is generated by a ti...

Let us Build Bridges: Understanding and Extending Diffusion Generative Models

Diffusion-based generative models have achieved promising results recent...

Linear dynamical neural population models through nonlinear embeddings

A body of recent work in modeling neural activity focuses on recovering ...