Kernel Stein Discrepancy Descent

by   Anna Korba, et al.

Among dissimilarities between probability distributions, the Kernel Stein Discrepancy (KSD) has received much interest recently. We investigate the properties of its Wasserstein gradient flow to approximate a target probability distribution π on ℝ^d, known up to a normalization constant. This leads to a straightforwardly implementable, deterministic score-based method to sample from π, named KSD Descent, which uses a set of particles to approximate π. Remarkably, owing to a tractable loss function, KSD Descent can leverage robust parameter-free optimization schemes such as L-BFGS; this contrasts with other popular particle-based schemes such as the Stein Variational Gradient Descent algorithm. We study the convergence properties of KSD Descent and demonstrate its practical relevance. However, we also highlight failure cases by showing that the algorithm can get stuck in spurious local minima.



page 1

page 2

page 3

page 4


A Non-Asymptotic Analysis for Stein Variational Gradient Descent

We study the Stein Variational Gradient Descent (SVGD) algorithm, which ...

On the geometry of Stein variational gradient descent

Bayesian inference problems require sampling or approximating high-dimen...

Unbalanced Sobolev Descent

We introduce Unbalanced Sobolev Descent (USD), a particle descent algori...

Kernel Selection for Stein Variational Gradient Descent

Stein variational gradient descent (SVGD) and its variants have shown pr...

Maximum Mean Discrepancy Gradient Flow

We construct a Wasserstein gradient flow of the maximum mean discrepancy...

Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization

We consider the optimization problem of minimizing a functional defined ...

De-randomizing MCMC dynamics with the diffusion Stein operator

Approximate Bayesian inference estimates descriptors of an intractable t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.