Towards Improved Learning in Gaussian Processes: The Best of Two Worlds

11/11/2022
by   Rui Li, et al.
0

Gaussian process training decomposes into inference of the (approximate) posterior and learning of the hyperparameters. For non-Gaussian (non-conjugate) likelihoods, two common choices for approximate inference are Expectation Propagation (EP) and Variational Inference (VI), which have complementary strengths and weaknesses. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, it does not automatically imply it is a good learning objective for hyperparameter optimization. We design a hybrid training procedure where the inference leverages conjugate-computation VI and the learning uses an EP-like marginal likelihood approximation. We empirically demonstrate on binary classification that this provides a good learning objective and generalizes better.

READ FULL TEXT

page 2

page 4

page 6

research
06/07/2023

Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Models

Approximate inference in Gaussian process (GP) models with non-conjugate...
research
12/21/2019

Quantile Propagation for Wasserstein-Approximate Gaussian Processes

In this work, we develop a new approximation method to solve the analyti...
research
12/03/2014

Nested Variational Compression in Deep Gaussian Processes

Deep Gaussian processes provide a flexible approach to probabilistic mod...
research
11/05/2021

Dual Parameterization of Sparse Variational Gaussian Processes

Sparse variational Gaussian process (SVGP) methods are a common choice f...
research
06/22/2017

Scalable Multi-Class Gaussian Process Classification using Expectation Propagation

This paper describes an expectation propagation (EP) method for multi-cl...
research
08/16/2018

Learning Invariances using the Marginal Likelihood

Generalising well in supervised learning tasks relies on correctly extra...
research
09/13/2018

Gaussian process classification using posterior linearisation

This paper proposes a new algorithm for Gaussian process classification ...

Please sign up or login with your details

Forgot password? Click here to reset