Stochastic Variational Inference for Fully Bayesian Sparse Gaussian Process Regression Models

11/01/2017
by   Haibin Yu, et al.
0

This paper presents a novel variational inference framework for deriving a family of Bayesian sparse Gaussian process regression (SGPR) models whose approximations are variationally optimal with respect to the full-rank GPR model enriched with various corresponding correlation structures of the observation noises. Our variational Bayesian SGPR (VBSGPR) models jointly treat both the distributions of the inducing variables and hyperparameters as variational parameters, which enables the decomposability of the variational lower bound that in turn can be exploited for stochastic optimization. Such a stochastic optimization involves iteratively following the stochastic gradient of the variational lower bound to improve its estimates of the optimal variational distributions of the inducing variables and hyperparameters (and hence the predictive distribution) of our VBSGPR models and is guaranteed to achieve asymptotic convergence to them. We show that the stochastic gradient is an unbiased estimator of the exact gradient and can be computed in constant time per iteration, hence achieving scalability to big data. We empirically evaluate the performance of our proposed framework on two real-world, massive datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/05/2019

Scalable Variational Bayesian Kernel Selection for Sparse Gaussian Process Regression

This paper presents a variational Bayesian kernel selection (VBKS) algor...
research
11/18/2016

A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression

While much research effort has been dedicated to scaling up sparse Gauss...
research
03/06/2020

Rethinking Sparse Gaussian Processes: Bayesian Approaches to Inducing-Variable Approximations

Variational inference techniques based on inducing variables provide an ...
research
06/23/2020

Variational Orthogonal Features

Sparse stochastic variational inference allows Gaussian process models t...
research
09/14/2019

Scalable Gaussian Process Classification with Additive Noise for Various Likelihoods

Gaussian process classification (GPC) provides a flexible and powerful s...
research
09/14/2016

Gray-box inference for structured Gaussian process models

We develop an automated variational inference method for Bayesian struct...
research
09/09/2015

Fast Second-Order Stochastic Backpropagation for Variational Inference

We propose a second-order (Hessian or Hessian-free) based optimization m...

Please sign up or login with your details

Forgot password? Click here to reset