Posterior and Computational Uncertainty in Gaussian Processes

05/30/2022
by   Jonathan Wenger, et al.
9

Gaussian processes scale prohibitively with the size of the dataset. In response, many approximation methods have been developed, which inevitably introduce approximation error. This additional source of uncertainty, due to limited computation, is entirely ignored when using the approximate posterior. Therefore in practice, GP models are often as much about the approximation method as they are about the data. Here, we develop a new class of methods that provides consistent estimation of the combined uncertainty arising from both the finite number of data observed and the finite amount of computation expended. The most common GP approximations map to an instance in this class, such as methods based on the Cholesky factorization, conjugate gradients, and inducing points. For any method in this class, we prove (i) convergence of its posterior mean in the associated RKHS, (ii) decomposability of its combined posterior covariance into mathematical and computational covariances, and (iii) that the combined variance is a tight worst-case bound for the squared error between the method's posterior mean and the latent function. Finally, we empirically demonstrate the consequences of ignoring computational uncertainty and show how implicitly modeling it improves generalization performance on benchmark datasets.

READ FULL TEXT

page 4

page 21

research
07/15/2021

Input Dependent Sparse Gaussian Processes

Gaussian Processes (GPs) are Bayesian models that provide uncertainty es...
research
10/10/2019

Deep Structured Mixtures of Gaussian Processes

Gaussian Processes (GPs) are powerful non-parametric Bayesian regression...
research
03/16/2018

Constant-Time Predictive Distributions for Gaussian Processes

One of the most compelling features of Gaussian process (GP) regression ...
research
06/04/2019

Posterior Variance Analysis of Gaussian Processes with Application to Average Learning Curves

The posterior variance of Gaussian processes is a valuable measure of th...
research
09/10/2022

Revisiting Active Sets for Gaussian Process Decoders

Decoders built on Gaussian processes (GPs) are enticing due to the margi...
research
05/08/2022

Sparse Gaussian processes for solving nonlinear PDEs

In this article, we propose a numerical method based on sparse Gaussian ...
research
05/27/2021

Deconditional Downscaling with Gaussian Processes

Refining low-resolution (LR) spatial fields with high-resolution (HR) in...

Please sign up or login with your details

Forgot password? Click here to reset