Annealed Stein Variational Gradient Descent

01/24/2021
by   Francesco D'Angelo, et al.
0

Particle based optimization algorithms have recently been developed as sampling methods that iteratively update a set of particles to approximate a target distribution. In particular Stein variational gradient descent has gained attention in the approximate inference literature for its flexibility and accuracy. We empirically explore the ability of this method to sample from multi-modal distributions and focus on two important issues: (i) the inability of the particles to escape from local modes and (ii) the inefficacy in reproducing the density of the different regions. We propose an annealing schedule to solve these issues and show, through various experiments, how this simple solution leads to significant improvements in mode coverage, without invalidating any theoretical properties of the original algorithm.

READ FULL TEXT

page 6

page 9

research
06/04/2022

Stochastic Multiple Target Sampling Gradient Descent

Sampling from an unnormalized target distribution is an essential proble...
research
05/20/2021

Kernel Stein Discrepancy Descent

Among dissimilarities between probability distributions, the Kernel Stei...
research
09/05/2018

Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory

Particle-optimization sampling (POS) is a recently developed technique t...
research
05/21/2018

Frank-Wolfe Stein Sampling

In Bayesian inference, the posterior distributions are difficult to obta...
research
05/18/2023

Augmented Message Passing Stein Variational Gradient Descent

Stein Variational Gradient Descent (SVGD) is a popular particle-based me...
research
10/27/2018

Stein Variational Gradient Descent as Moment Matching

Stein variational gradient descent (SVGD) is a non-parametric inference ...
research
10/27/2020

Particle gradient descent model for point process generation

This paper introduces a generative model for planar point processes in a...

Please sign up or login with your details

Forgot password? Click here to reset