Multilevel Stein variational gradient descent with applications to Bayesian inverse problems

04/05/2021
by   Terrence Alsup, et al.
0

This work presents a multilevel variant of Stein variational gradient descent to more efficiently sample from target distributions. The key ingredient is a sequence of distributions with growing fidelity and costs that converges to the target distribution of interest. For example, such a sequence of distributions is given by a hierarchy of ever finer discretization levels of the forward model in Bayesian inverse problems. The proposed multilevel Stein variational gradient descent moves most of the iterations to lower, cheaper levels with the aim of requiring only a few iterations on the higher, more expensive levels when compared to the traditional, single-level Stein variational gradient descent variant that uses the highest-level distribution only. Under certain assumptions, in the mean-field limit, the error of the proposed multilevel Stein method decays by a log factor faster than the error of the single-level counterpart with respect to computational costs. Numerical experiments with Bayesian inverse problems show speedups of more than one order of magnitude of the proposed multilevel Stein method compared to the single-level variant that uses the highest level only.

READ FULL TEXT

page 12

page 20

research
12/06/2022

Further analysis of multilevel Stein variational gradient descent with an application to the Bayesian inference of glacier ice models

Multilevel Stein variational gradient descent is a method for particle-b...
research
04/28/2022

Multilevel Optimization for Inverse Problems

Inverse problems occur in a variety of parameter identification tasks in...
research
05/28/2021

A Gradient Method for Multilevel Optimization

Although application examples of multilevel optimization have already be...
research
03/07/2019

Solutions to Sparse Multilevel Matrix Problems

We define and solve classes of sparse matrix problems that arise in mult...
research
04/27/2019

Multilevel adaptive sparse Leja approximations for Bayesian inverse problems

Deterministic interpolation and quadrature methods are often unsuitable ...
research
02/08/2022

Multilevel Delayed Acceptance MCMC

We develop a novel Markov chain Monte Carlo (MCMC) method that exploits ...
research
04/13/2021

Stein variational gradient descent with local approximations

Bayesian computation plays an important role in modern machine learning ...

Please sign up or login with your details

Forgot password? Click here to reset