Multilevel Stein variational gradient descent with applications to Bayesian inverse problems

by   Terrence Alsup, et al.

This work presents a multilevel variant of Stein variational gradient descent to more efficiently sample from target distributions. The key ingredient is a sequence of distributions with growing fidelity and costs that converges to the target distribution of interest. For example, such a sequence of distributions is given by a hierarchy of ever finer discretization levels of the forward model in Bayesian inverse problems. The proposed multilevel Stein variational gradient descent moves most of the iterations to lower, cheaper levels with the aim of requiring only a few iterations on the higher, more expensive levels when compared to the traditional, single-level Stein variational gradient descent variant that uses the highest-level distribution only. Under certain assumptions, in the mean-field limit, the error of the proposed multilevel Stein method decays by a log factor faster than the error of the single-level counterpart with respect to computational costs. Numerical experiments with Bayesian inverse problems show speedups of more than one order of magnitude of the proposed multilevel Stein method compared to the single-level variant that uses the highest level only.



There are no comments yet.


page 12

page 20


Multilevel adaptive sparse Leja approximations for Bayesian inverse problems

Deterministic interpolation and quadrature methods are often unsuitable ...

Stein variational reduced basis Bayesian inversion

We propose and analyze a Stein variational reduced basis method (SVRB) t...

A Gradient Method for Multilevel Optimization

Although application examples of multilevel optimization have already be...

Solutions to Sparse Multilevel Matrix Problems

We define and solve classes of sparse matrix problems that arise in mult...

On the geometry of Stein variational gradient descent

Bayesian inference problems require sampling or approximating high-dimen...

Multilevel Dimension-Independent Likelihood-Informed MCMC for Large-Scale Inverse Problems

We present a non-trivial integration of dimension-independent likelihood...

Stein variational gradient descent with local approximations

Bayesian computation plays an important role in modern machine learning ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.