Efficiently sampling functions from Gaussian process posteriors

02/21/2020
by   James T. Wilson, et al.
4

Gaussian processes are the gold standard for many real-world modeling problems, especially in cases where a model's success hinges upon its ability to faithfully represent predictive uncertainty. These problems typically exist as parts of larger frameworks, where quantities of interest are ultimately defined by integrating over posterior distributions. However, these algorithms' inner workings rarely allow for closed-form integration, giving rise to a need for Monte Carlo methods. Despite substantial progress in scaling up Gaussian processes to large training sets, methods for accurately generating draws from their posterior distributions still scale cubically in the number of test locations. We identify a decomposition of Gaussian processes that naturally lends itself to scalable sampling by enabling us to efficiently generate functions that accurately represent their posteriors. Building off of this factorization, we propose decoupled sampling, an easy-to-use and general-purpose approach for fast posterior sampling. Decoupled sampling works as a drop-in strategy that seamlessly pairs with sparse approximations to Gaussian processes to afford scalability both during training and at test time. In a series of experiments designed to test competing sampling schemes' statistical behaviors and practical ramifications, we empirically show that functions drawn using decoupled sampling faithfully represent Gaussian process posteriors at a fraction of the usual cost.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/08/2020

Pathwise Conditioning of Gaussian Processes

As Gaussian processes are integrated into increasingly complex problem s...
research
06/14/2020

GP3: A Sampling-based Analysis Framework for Gaussian Processes

Although machine learning is increasingly applied in control approaches,...
research
10/30/2019

Bayesian nonstationary Gaussian process modeling: the BayesNSGP package for R

In spite of the diverse literature on nonstationary Gaussian process mod...
research
02/28/2022

GPU-Accelerated Policy Optimization via Batch Automatic Differentiation of Gaussian Processes for Real-World Control

The ability of Gaussian processes (GPs) to predict the behavior of dynam...
research
06/08/2022

Neural Diffusion Processes

Gaussian processes provide an elegant framework for specifying prior and...
research
12/12/2020

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

Skew-Gaussian processes (SkewGPs) extend the multivariate Unified Skew-N...
research
05/24/2018

Log Gaussian Cox Process Networks

We generalize the log Gaussian Cox process (LGCP) framework to model mul...

Please sign up or login with your details

Forgot password? Click here to reset