Approximate Inference for Spectral Mixture Kernel

06/12/2020
by   Yohan Jung, et al.
0

A spectral mixture (SM) kernel is a flexible kernel used to model any stationary covariance function. Although it is useful in modeling data, the learning of the SM kernel is generally difficult because optimizing a large number of parameters for the SM kernel typically induces an over-fitting, particularly when a gradient-based optimization is used. Also, a longer training time is required. To improve the training, we propose an approximate Bayesian inference for the SM kernel. Specifically, we employ the variational distribution of the spectral points to approximate SM kernel with a random Fourier feature. We optimize the variational parameters by applying a sampling-based variational inference to the derived evidence lower bound (ELBO) estimator constructed from the approximate kernel. To improve the inference, we further propose two additional strategies: (1) a sampling strategy of spectral points to estimate the ELBO estimator reliably and thus its associated gradient, and (2) an approximate natural gradient to accelerate the convergence of the parameters. The proposed inference combined with two strategies accelerates the convergence of the parameters and leads to better optimal parameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/22/2021

Neural Variational Gradient Descent

Particle-based approximate Bayesian inference approaches such as Stein V...
research
07/29/2020

Approximation Based Variance Reduction for Reparameterization Gradients

Flexible variational distributions improve variational inference but are...
research
11/04/2020

Quantized Variational Inference

We present Quantized Variational Inference, a new algorithm for Evidence...
research
07/12/2018

Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models

Bayesian inference plays an important role in advancing machine learning...
research
07/06/2020

Kernel Stein Generative Modeling

We are interested in gradient-based Explicit Generative Modeling where s...
research
06/07/2018

A Spectral Approach to Gradient Estimation for Implicit Distributions

Recently there have been increasing interests in learning and inference ...
research
03/13/2019

Generalized Elliptical Slice Sampling with Regional Pseudo-priors

In this paper, we propose a MCMC algorithm based on elliptical slice sam...

Please sign up or login with your details

Forgot password? Click here to reset