Kernel Selection for Stein Variational Gradient Descent

07/20/2021
by   Qingzhong Ai, et al.
0

Stein variational gradient descent (SVGD) and its variants have shown promising successes in approximate inference for complex distributions. However, their empirical performance depends crucially on the choice of optimal kernel. Unfortunately, RBF kernel with median heuristics is a common choice in previous approaches which has been proved sub-optimal. Inspired by the paradigm of multiple kernel learning, our solution to this issue is using a combination of multiple kernels to approximate the optimal kernel instead of a single one which may limit the performance and flexibility. To do so, we extend Kernelized Stein Discrepancy (KSD) to its multiple kernel view called Multiple Kernelized Stein Discrepancy (MKSD). Further, we leverage MKSD to construct a general algorithm based on SVGD, which be called Multiple Kernel SVGD (MK-SVGD). Besides, we automatically assign a weight to each kernel without any other parameters. The proposed method not only gets rid of optimal kernel dependence but also maintains computational effectiveness. Experiments on various tasks and models show the effectiveness of our method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/22/2021

Neural Variational Gradient Descent

Particle-based approximate Bayesian inference approaches such as Stein V...
research
06/30/2020

Sliced Kernelized Stein Discrepancy

Kernelized Stein discrepancy (KSD), though being extensively used in goo...
research
05/20/2021

Kernel Stein Discrepancy Descent

Among dissimilarities between probability distributions, the Kernel Stei...
research
02/05/2021

Active Slices for Sliced Stein Discrepancy

Sliced Stein discrepancy (SSD) and its kernelized variants have demonstr...
research
12/16/2011

Insights from Classifying Visual Concepts with Multiple Kernel Learning

Combining information from various image features has become a standard ...
research
06/07/2018

Stein Variational Gradient Descent Without Gradient

Stein variational gradient decent (SVGD) has been shown to be a powerful...
research
10/27/2018

Stein Variational Gradient Descent as Moment Matching

Stein variational gradient descent (SVGD) is a non-parametric inference ...

Please sign up or login with your details

Forgot password? Click here to reset