Sparse Algorithms for Markovian Gaussian Processes

by   William J. Wilkinson, et al.

Approximate Bayesian inference methods that scale to very large datasets are crucial in leveraging probabilistic models for real-world time series. Sparse Markovian Gaussian processes combine the use of inducing variables with efficient Kalman filter-like recursions, resulting in algorithms whose computational and memory requirements scale linearly in the number of inducing points, whilst also enabling parallel parameter updates and stochastic optimisation. Under this paradigm, we derive a general site-based approach to approximate inference, whereby we approximate the non-Gaussian likelihood with local Gaussian terms, called sites. Our approach results in a suite of novel sparse extensions to algorithms from both the machine learning and signal processing literature, including variational inference, expectation propagation, and the classical nonlinear Kalman smoothers. The derived methods are suited to large time series, and we also demonstrate their applicability to spatio-temporal data, where the model has separate inducing points in both time and space.



page 7


Blitzkriging: Kronecker-structured Stochastic Gaussian Processes

We present Blitzkriging, a new approach to fast inference for Gaussian p...

State Space Expectation Propagation: Efficient Inference Schemes for Temporal Gaussian Processes

We formulate approximate Bayesian inference in non-conjugate temporal an...

Probabilistic selection of inducing points in sparse Gaussian processes

Sparse Gaussian processes and various extensions thereof are enabled thr...

Bayes-Newton Methods for Approximate Bayesian Inference with PSD Guarantees

We formulate natural gradient variational inference (VI), expectation pr...

Sparse within Sparse Gaussian Processes using Neighbor Information

Approximations to Gaussian processes based on inducing variables, combin...

Compositional Modeling of Nonlinear Dynamical Systems with ODE-based Random Features

Effectively modeling phenomena present in highly nonlinear dynamical sys...

SigGPDE: Scaling Sparse Gaussian Processes on Sequential Data

Making predictions and quantifying their uncertainty when the input data...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.