Rethinking Sparse Gaussian Processes: Bayesian Approaches to Inducing-Variable Approximations

03/06/2020
by   Simone Rossi, et al.
14

Variational inference techniques based on inducing variables provide an elegant framework for scalable posterior estimation in Gaussian process (GP) models. Most previous works treat the locations of the inducing variables, i.e. the inducing inputs, as variational hyperparameters, and these are then optimized together with GP covariance hyper-parameters. While some approaches point to the benefits of a Bayesian treatment of GP hyper-parameters, this has been largely overlooked for the inducing inputs. In this work, we show that treating both inducing locations and GP hyper-parameters in a Bayesian way, by inferring their full posterior, further significantly improves performance. Based on stochastic gradient Hamiltonian Monte Carlo, we develop a fully Bayesian approach to scalable GP and deep GP models, and demonstrate its competitive performance through an extensive experimental campaign across several regression and classification problems.

READ FULL TEXT
research
10/19/2017

Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition

We propose a method (TT-GP) for approximate inference in Gaussian Proces...
research
06/12/2015

MCMC for Variationally Sparse Gaussian Processes

Gaussian process (GP) models form a core part of probabilistic machine l...
research
11/01/2017

Stochastic Variational Inference for Fully Bayesian Sparse Gaussian Process Regression Models

This paper presents a novel variational inference framework for deriving...
research
10/31/2019

Continual Multi-task Gaussian Processes

We address the problem of continual learning in multi-task Gaussian proc...
research
07/06/2018

Fully Scalable Gaussian Processes using Subspace Inducing Inputs

We introduce fully scalable Gaussian processes, an implementation scheme...
research
04/27/2015

On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes

The variational framework for learning inducing variables (Titsias, 2009...
research
05/16/2019

The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions

Discovering interaction effects on a response of interest is a fundament...

Please sign up or login with your details

Forgot password? Click here to reset