Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling

06/01/2021
by   Valentin De Bortoli, et al.
21

Progressively applying Gaussian noise transforms complex data distributions to approximately Gaussian. Reversing this dynamic defines a generative model. When the forward noising process is given by a Stochastic Differential Equation (SDE), Song et al. (2021) demonstrate how the time inhomogeneous drift of the associated reverse-time SDE may be estimated using score-matching. A limitation of this approach is that the forward-time SDE must be run for a sufficiently long time for the final distribution to be approximately Gaussian. In contrast, solving the Schrödinger Bridge problem (SB), i.e. an entropy-regularized optimal transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time. We present Diffusion SB (DSB), an original approximation of the Iterative Proportional Fitting (IPF) procedure to solve the SB problem, and provide theoretical analysis along with generative modeling experiments. The first DSB iteration recovers the methodology proposed by Song et al. (2021), with the flexibility of using shorter time intervals, as subsequent DSB iterations reduce the discrepancy between the final-time marginal of the forward (resp. backward) SDE with respect to the prior (resp. data) distribution. Beyond generative modeling, DSB offers a widely applicable computational optimal transport tool as the continuous state-space analogue of the popular Sinkhorn algorithm (Cuturi, 2013).

READ FULL TEXT
research
03/29/2023

Diffusion Schrödinger Bridge Matching

Solving transport problems, i.e. finding a map transporting one given di...
research
04/03/2023

Diffusion Bridge Mixture Transports, Schrödinger Bridge Problems and Generative Modeling

The dynamic Schrödinger bridge problem seeks a stochastic process that d...
research
04/11/2023

Generative modeling for time series via Schrödinger bridge

We propose a novel generative model for time series based on Schrödinger...
research
10/21/2021

Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory

Schrödinger Bridge (SB) is an optimal transport problem that has receive...
research
08/16/2017

On residual and guided proposals for diffusion bridge simulation

Recently Whitaker et al. (2017) considered Bayesian estimation of diffus...
research
07/03/2023

Transport, Variational Inference and Diffusions: with Applications to Annealed Flows and Schrödinger Bridges

This paper explores the connections between optimal transport and variat...
research
08/09/2022

Wavelet Score-Based Generative Modeling

Score-based generative models (SGMs) synthesize new data samples from Ga...

Please sign up or login with your details

Forgot password? Click here to reset