Kernel Stein Discrepancy Descent

05/20/2021
by   Anna Korba, et al.
0

Among dissimilarities between probability distributions, the Kernel Stein Discrepancy (KSD) has received much interest recently. We investigate the properties of its Wasserstein gradient flow to approximate a target probability distribution π on ℝ^d, known up to a normalization constant. This leads to a straightforwardly implementable, deterministic score-based method to sample from π, named KSD Descent, which uses a set of particles to approximate π. Remarkably, owing to a tractable loss function, KSD Descent can leverage robust parameter-free optimization schemes such as L-BFGS; this contrasts with other popular particle-based schemes such as the Stein Variational Gradient Descent algorithm. We study the convergence properties of KSD Descent and demonstrate its practical relevance. However, we also highlight failure cases by showing that the algorithm can get stuck in spurious local minima.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

06/17/2020

A Non-Asymptotic Analysis for Stein Variational Gradient Descent

We study the Stein Variational Gradient Descent (SVGD) algorithm, which ...
12/02/2019

On the geometry of Stein variational gradient descent

Bayesian inference problems require sampling or approximating high-dimen...
09/29/2020

Unbalanced Sobolev Descent

We introduce Unbalanced Sobolev Descent (USD), a particle descent algori...
07/20/2021

Kernel Selection for Stein Variational Gradient Descent

Stein variational gradient descent (SVGD) and its variants have shown pr...
06/11/2019

Maximum Mean Discrepancy Gradient Flow

We construct a Wasserstein gradient flow of the maximum mean discrepancy...
12/21/2020

Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization

We consider the optimization problem of minimizing a functional defined ...
10/07/2021

De-randomizing MCMC dynamics with the diffusion Stein operator

Approximate Bayesian inference estimates descriptors of an intractable t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.