Understanding and Comparing Scalable Gaussian Process Regression for Big Data

11/03/2018
by   Haitao Liu, et al.
0

As a non-parametric Bayesian model which produces informative predictive distribution, Gaussian process (GP) has been widely used in various fields, like regression, classification and optimization. The cubic complexity of standard GP however leads to poor scalability, which poses challenges in the era of big data. Hence, various scalable GPs have been developed in the literature in order to improve the scalability while retaining desirable prediction accuracy. This paper devotes to investigating the methodological characteristics and performance of representative global and local scalable GPs including sparse approximations and local aggregations from four main perspectives: scalability, capability, controllability and robustness. The numerical experiments on two toy examples and five real-world datasets with up to 250K points offer the following findings. In terms of scalability, most of the scalable GPs own a time complexity that is linear to the training size. In terms of capability, the sparse approximations capture the long-term spatial correlations, the local aggregations capture the local patterns but suffer from over-fitting in some scenarios. In terms of controllability, we could improve the performance of sparse approximations by simply increasing the inducing size. But this is not the case for local aggregations. In terms of robustness, local aggregations are robust to various initializations of hyperparameters due to the local attention mechanism. Finally, we highlight that the proper hybrid of global and local scalable GPs may be a promising way to improve both the model capability and scalability for big data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/03/2018

When Gaussian Process Meets Big Data: A Review of Scalable GPs

The vast quantity of information brought by big data as well as the evol...
research
11/03/2018

Large-scale Heteroscedastic Regression via Gaussian Process

Heteroscedastic regression which considers varying noises across input d...
research
08/09/2014

Parallel Gaussian Process Regression with Low-Rank Covariance Matrix Approximations

Gaussian processes (GP) are Bayesian non-parametric models that are wide...
research
03/03/2020

Multi-Sparse Gaussian Process: Learning based Semi-Parametric Control

A key challenge with controlling complex dynamical systems is to accurat...
research
11/17/2014

Parallel Gaussian Process Regression for Big Data: Low-Rank Representation Meets Markov Approximation

The expressive power of a Gaussian process (GP) model comes at a cost of...
research
05/27/2020

Precision Aggregated Local Models

Large scale Gaussian process (GP) regression is infeasible for larger da...
research
03/30/2020

Representing Realistic Human Driver Behaviors using a Finite Size Gaussian Process Kernel Bank

The performance of cooperative vehicular applications is tightly depende...

Please sign up or login with your details

Forgot password? Click here to reset