Precision Aggregated Local Models

by   Adam M. Edwards, et al.

Large scale Gaussian process (GP) regression is infeasible for larger data sets due to cubic scaling of flops and quadratic storage involved in working with covariance matrices. Remedies in recent literature focus on divide-and-conquer, e.g., partitioning into sub-problems and inducing functional (and thus computational) independence. Such approximations can be speedy, accurate, and sometimes even more flexible than an ordinary GPs. However, a big downside is loss of continuity at partition boundaries. Modern methods like local approximate GPs (LAGPs) imply effectively infinite partitioning and are thus pathologically good and bad in this regard. Model averaging, an alternative to divide-and-conquer, can maintain absolute continuity but often over-smooths, diminishing accuracy. Here we propose putting LAGP-like methods into a local experts-like framework, blending partition-based speed with model-averaging continuity, as a flagship example of what we call precision aggregated local models (PALM). Using K LAGPs, each selecting n from N total data pairs, we illustrate a scheme that is at most cubic in n, quadratic in K, and linear in N, drastically reducing computational and storage demands. Extensive empirical illustration shows how PALM is at least as accurate as LAGP, can be much faster in terms of speed, and furnishes continuous predictive surfaces. Finally, we propose sequential updating scheme which greedily refines a PALM predictor up to a computational budget.


page 5

page 6

page 8

page 11

page 12

page 20

page 23

page 26


Fast increased fidelity approximate Gibbs samplers for Bayesian Gaussian process regression

The use of Gaussian processes (GPs) is supported by efficient sampling a...

When Gaussian Process Meets Big Data: A Review of Scalable GPs

The vast quantity of information brought by big data as well as the evol...

Recursive Estimation for Sparse Gaussian Process Regression

Gaussian Processes (GPs) are powerful kernelized methods for non-paramet...

Understanding and Comparing Scalable Gaussian Process Regression for Big Data

As a non-parametric Bayesian model which produces informative predictive...

Patchwork Kriging for Large-scale Gaussian Process Regression

This paper presents a new approach for Gaussian process (GP) regression ...

Sparse Pseudo-input Local Kriging for Large Non-stationary Spatial Datasets with Exogenous Variables

Gaussian process (GP) regression is a powerful tool for building predict...

Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression

Inspired by recent advances in the field of expert-based approximations ...

Please sign up or login with your details

Forgot password? Click here to reset