Learning Deep Mixtures of Gaussian Process Experts Using Sum-Product Networks

09/12/2018
by   Martin Trapp, et al.
0

While Gaussian processes (GPs) are the method of choice for regression tasks, they also come with practical difficulties, as inference cost scales cubic in time and quadratic in memory. In this paper, we introduce a natural and expressive way to tackle these problems, by incorporating GPs in sum-product networks (SPNs), a recently proposed tractable probabilistic model allowing exact and efficient inference. In particular, by using GPs as leaves of an SPN we obtain a novel flexible prior over functions, which implicitly represents an exponentially large mixture of local GPs. Exact and efficient posterior inference in this model can be done in a natural interplay of the inference mechanisms in GPs and SPNs. Thereby, each GP is -- similarly as in a mixture of experts approach -- responsible only for a subset of data points, which effectively reduces inference cost in a divide and conquer fashion. We show that integrating GPs into the SPN framework leads to a promising probabilistic regression model which is: (1) computational and memory efficient, (2) allows efficient and exact posterior inference, (3) is flexible enough to mix different kernel functions, and (4) naturally accounts for non-stationarities in time series. In a variate of experiments, we show that the SPN-GP model can learn input dependent parameters and hyper-parameters and is on par with or outperforms the traditional GPs as well as state of the art approximations on real-world data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/10/2019

Deep Structured Mixtures of Gaussian Processes

Gaussian Processes (GPs) are powerful non-parametric Bayesian regression...
research
08/16/2011

Overlapping Mixtures of Gaussian Processes for the Data Association Problem

In this work we introduce a mixture of GPs to address the data associati...
research
07/03/2018

When Gaussian Process Meets Big Data: A Review of Scalable GPs

The vast quantity of information brought by big data as well as the evol...
research
12/17/2021

Correlated Product of Experts for Sparse Gaussian Process Regression

Gaussian processes (GPs) are an important tool in machine learning and s...
research
06/16/2021

Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression

Inspired by recent advances in the field of expert-based approximations ...
research
01/21/2023

Scalable Gaussian Process Inference with Stan

Gaussian processes (GPs) are sophisticated distributions to model functi...
research
07/04/2018

Neural Processes

A neural network (NN) is a parameterised function that can be tuned via ...

Please sign up or login with your details

Forgot password? Click here to reset