Log In Sign Up

Mixtures of Gaussian Process Experts with SMC^2

by   Teemu Härkönen, et al.

Gaussian processes are a key component of many flexible statistical and machine learning models. However, they exhibit cubic computational complexity and high memory constraints due to the need of inverting and storing a full covariance matrix. To circumvent this, mixtures of Gaussian process experts have been considered where data points are assigned to independent experts, reducing the complexity by allowing inference based on smaller, local covariance matrices. Moreover, mixtures of Gaussian process experts substantially enrich the model's flexibility, allowing for behaviors such as non-stationarity, heteroscedasticity, and discontinuities. In this work, we construct a novel inference approach based on nested sequential Monte Carlo samplers to simultaneously infer both the gating network and Gaussian process expert parameters. This greatly improves inference compared to importance sampling, particularly in settings when a stationary Gaussian process is inappropriate, while still being thoroughly parallelizable.


page 19

page 20

page 22

page 23


Infinite Mixtures of Multivariate Gaussian Processes

This paper presents a new model called infinite mixtures of multivariate...

Nonstationary Gaussian Process Emulators with Kernel Mixtures

Weakly stationary Gaussian processes are the principal tool in the stati...

Semi-Supervised Non-Parametric Bayesian Modelling of Spatial Proteomics

Understanding sub-cellular protein localisation is an essential componen...

Enriched Mixtures of Gaussian Process Experts

Mixtures of experts probabilistically divide the input space into region...

Probabilistic approach to limited-data computed tomography reconstruction

We consider the problem of reconstructing the internal structure of an o...

High-Dimensional Gaussian Process Inference with Derivatives

Although it is widely known that Gaussian processes can be conditioned o...

Infrastructure Recovery Curve Estimation Using Gaussian Process Regression on Expert Elicited Data

Infrastructure recovery time estimation is critical to disaster manageme...