Langevin Diffusion Variational Inference

08/16/2022
by   Tomas Geffner, et al.
0

Many methods that build powerful variational distributions based on unadjusted Langevin transitions exist. Most of these were developed using a wide range of different approaches and techniques. Unfortunately, the lack of a unified analysis and derivation makes developing new methods and reasoning about existing ones a challenging task. We address this giving a single analysis that unifies and generalizes these existing techniques. The main idea is to augment the target and variational by numerically simulating the underdamped Langevin diffusion process and its time reversal. The benefits of this approach are twofold: it provides a unified formulation for many existing methods, and it simplifies the development of new ones. In fact, using our formulation we propose a new method that combines the strengths of previously existing algorithms; it uses underdamped Langevin transitions and powerful augmentations parameterized by a score network. Our empirical evaluation shows that our proposed method consistently outperforms relevant baselines in a wide range of tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/14/2021

Simulating Diffusion Bridges with Score Matching

We consider the problem of simulating diffusion bridges, i.e. diffusion ...
research
02/09/2018

Black-box Variational Inference for Stochastic Differential Equations

Parameter inference for stochastic differential equations is challenging...
research
01/30/2019

Probability Functional Descent: A Unifying Perspective on GANs, Variational Inference, and Reinforcement Learning

The goal of this paper is to provide a unifying view of a wide range of ...
research
05/27/2019

Walsh-Hadamard Variational Inference for Bayesian Deep Learning

Over-parameterized models, such as DeepNets and ConvNets, form a class o...
research
10/14/2022

On the Relationship Between Variational Inference and Auto-Associative Memory

In this article, we propose a variational inference formulation of auto-...
research
06/11/2018

Hyperviscosity-Based Stabilization for Radial Basis Function-Finite Difference (RBF-FD) Discretizations of Advection-Diffusion Equations

We present a novel hyperviscosity formulation for stabilizing RBF-FD dis...

Please sign up or login with your details

Forgot password? Click here to reset