Regularized Sparse Gaussian Processes

10/13/2019
by   Rui Meng, et al.
0

Gaussian processes are a flexible Bayesian nonparametric modelling approach that has been widely applied to learning tasks such as facial expression recognition, image reconstruction, and human pose estimation. To address the issues of poor scaling from exact inference methods, approximation methods based on sparse Gaussian processes (SGP) and variational inference (VI) are necessary for the inference on large datasets. However, one of the problems involved in SGP, especially in latent variable models, is that the distribution of the inducing inputs may fail to capture the distribution of training inputs, which may lead to inefficient inference and poor model prediction. Hence, we propose a regularization approach for sparse Gaussian processes. We also extend this regularization approach into latent sparse Gaussian processes in a unified view, considering the balance of the distribution of inducing inputs and embedding inputs. Furthermore, we justify that performing VI on a sparse latent Gaussian process with this regularization term is equivalent to performing VI on a related empirical Bayes model with a prior on the inducing inputs. Also stochastic variational inference is available for our regularization approach. Finally, the feasibility of our proposed regularization method is demonstrated on three real-world datasets.

READ FULL TEXT

page 5

page 7

research
10/19/2020

Probabilistic selection of inducing points in sparse Gaussian processes

Sparse Gaussian processes and various extensions thereof are enabled thr...
research
03/13/2018

Variational zero-inflated Gaussian processes with sparse kernels

Zero-inflated datasets, which have an excess of zero outputs, are common...
research
11/20/2015

Recurrent Gaussian Processes

We define Recurrent Gaussian Processes (RGP) models, a general family of...
research
10/27/2015

Blitzkriging: Kronecker-structured Stochastic Gaussian Processes

We present Blitzkriging, a new approach to fast inference for Gaussian p...
research
06/26/2021

Scalable Gaussian Processes for Data-Driven Design using Big Data with Categorical Factors

Scientific and engineering problems often require the use of artificial ...
research
02/06/2014

Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models - a Gentle Tutorial

In this tutorial we explain the inference procedures developed for the s...
research
05/22/2018

Variational Learning on Aggregate Outputs with Gaussian Processes

While a typical supervised learning framework assumes that the inputs an...

Please sign up or login with your details

Forgot password? Click here to reset