Deep Structured Mixtures of Gaussian Processes

by   Martin Trapp, et al.

Gaussian Processes (GPs) are powerful non-parametric Bayesian regression models that allow exact posterior inference, but exhibit high computational and memory costs. In order to improve scalability of GPs, approximate posterior inference is frequently employed, where a prominent class of approximation techniques is based on local GP experts. However, the local-expert techniques proposed so far are either not well-principled, come with limited approximation guarantees, or lead to intractable models. In this paper, we introduce deep structured mixtures of GP experts, a stochastic process model which i) allows exact posterior inference, ii) has attractive computational and memory costs, and iii), when used as GP approximation, captures predictive uncertainties consistently better than previous approximations. In a variety of experiments, we show that deep structured mixtures have a low approximation error and outperform existing expert-based approaches.


page 1

page 2

page 3

page 4


Learning Deep Mixtures of Gaussian Process Experts Using Sum-Product Networks

While Gaussian processes (GPs) are the method of choice for regression t...

On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach

Gaussian processes (GPs) are frequently used in machine learning and sta...

Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression

Inspired by recent advances in the field of expert-based approximations ...

Overlapping Mixtures of Gaussian Processes for the Data Association Problem

In this work we introduce a mixture of GPs to address the data associati...

Posterior and Computational Uncertainty in Gaussian Processes

Gaussian processes scale prohibitively with the size of the dataset. In ...

Enriched Mixtures of Gaussian Process Experts

Mixtures of experts probabilistically divide the input space into region...

When Gaussian Process Meets Big Data: A Review of Scalable GPs

The vast quantity of information brought by big data as well as the evol...