Mixtures of Gaussian Process Experts with SMC^2

08/26/2022
by   Teemu Härkönen, et al.
1

Gaussian processes are a key component of many flexible statistical and machine learning models. However, they exhibit cubic computational complexity and high memory constraints due to the need of inverting and storing a full covariance matrix. To circumvent this, mixtures of Gaussian process experts have been considered where data points are assigned to independent experts, reducing the complexity by allowing inference based on smaller, local covariance matrices. Moreover, mixtures of Gaussian process experts substantially enrich the model's flexibility, allowing for behaviors such as non-stationarity, heteroscedasticity, and discontinuities. In this work, we construct a novel inference approach based on nested sequential Monte Carlo samplers to simultaneously infer both the gating network and Gaussian process expert parameters. This greatly improves inference compared to importance sampling, particularly in settings when a stationary Gaussian process is inappropriate, while still being thoroughly parallelizable.

READ FULL TEXT

page 19

page 20

page 22

page 23

research
02/09/2023

Gaussian Process-Gated Hierarchical Mixtures of Experts

In this paper, we propose novel Gaussian process-gated hierarchical mixt...
research
03/07/2019

Semi-Supervised Non-Parametric Bayesian Modelling of Spatial Proteomics

Understanding sub-cellular protein localisation is an essential componen...
research
03/13/2018

Nonstationary Gaussian Process Emulators with Kernel Mixtures

Weakly stationary Gaussian processes are the principal tool in the stati...
research
05/30/2019

Enriched Mixtures of Gaussian Process Experts

Mixtures of experts probabilistically divide the input space into region...
research
09/11/2018

Probabilistic approach to limited-data computed tomography reconstruction

We consider the problem of reconstructing the internal structure of an o...
research
07/26/2013

Infinite Mixtures of Multivariate Gaussian Processes

This paper presents a new model called infinite mixtures of multivariate...
research
02/15/2021

High-Dimensional Gaussian Process Inference with Derivatives

Although it is widely known that Gaussian processes can be conditioned o...

Please sign up or login with your details

Forgot password? Click here to reset