DeepAI AI Chat
Log In Sign Up

A Non-Asymptotic Analysis for Stein Variational Gradient Descent

by   Anna Korba, et al.

We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution π∝ e^-V on R^d. In the population limit, SVGD performs gradient descent in the space of probability distributions on the KL divergence with respect to π, where the gradient is smoothed through a kernel integral operator. In this paper, we provide a novel finite time analysis for the SVGD algorithm. We obtain a descent lemma establishing that the algorithm decreases the objective at each iteration, and provably converges, with less restrictive assumptions on the step size than required in earlier analyses. We further provide a guarantee on the convergence rate in Kullback-Leibler divergence, assuming π satisfies a Stein log-Sobolev inequality as in Duncan et al. (2019), which takes into account the geometry induced by the smoothed KL gradient.


page 1

page 2

page 3

page 4


Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Condition

Stein Variational Gradient Descent (SVGD) is an important alternative to...

Stein Variational Gradient Descent as Gradient Flow

Stein variational gradient descent (SVGD) is a deterministic sampling al...

A Finite-Particle Convergence Rate for Stein Variational Gradient Descent

We provide a first finite-particle convergence rate for Stein variationa...

Kernel Stein Discrepancy Descent

Among dissimilarities between probability distributions, the Kernel Stei...

Sampling with Mirrored Stein Operators

We introduce a new family of particle evolution samplers suitable for co...

Riemannian optimization for non-centered mixture of scaled Gaussian distributions

This paper studies the statistical model of the non-centered mixture of ...

Code Repositories