Shallow and Deep Nonparametric Convolutions for Gaussian Processes

06/17/2022
by   Thomas M. McDonald, et al.
0

A key challenge in the practical application of Gaussian processes (GPs) is selecting a proper covariance function. The moving average, or process convolutions, construction of GPs allows some additional flexibility, but still requires choosing a proper smoothing kernel, which is non-trivial. Previous approaches have built covariance functions by using GP priors over the smoothing kernel, and by extension the covariance, as a way to bypass the need to specify it in advance. However, such models have been limited in several ways: they are restricted to single dimensional inputs, e.g. time; they only allow modelling of single outputs and they do not scale to large datasets since inference is not straightforward. In this paper, we introduce a nonparametric process convolution formulation for GPs that alleviates these weaknesses by using a functional sampling approach based on Matheron's rule to perform fast sampling using interdomain inducing variables. Furthermore, we propose a composition of these nonparametric convolutions that serves as an alternative to classic deep GP models, and allows the covariance functions of the intermediate layers to be inferred from the data. We test the performance of our model on benchmarks for single output GPs, multiple output GPs and deep GPs and find that in many cases our approach can provide improvements over standard GP models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/10/2021

Learning Nonparametric Volterra Kernels with Gaussian Processes

This paper introduces a method for the nonparametric Bayesian learning o...
research
03/05/2019

Emulating computer models with step-discontinuous outputs using Gaussian processes

In many real-world applications, we are interested in approximating func...
research
05/30/2017

Large Linear Multi-output Gaussian Process Learning

Gaussian processes (GPs), or distributions over arbitrary functions in a...
research
11/05/2015

Thoughts on Massively Scalable Gaussian Processes

We introduce a framework and early results for massively scalable Gaussi...
research
10/14/2020

Diffusion Based Gaussian Processes on Restricted Domains

In nonparametric regression and spatial process modeling, it is common f...
research
06/16/2021

Leveraging Probabilistic Circuits for Nonparametric Multi-Output Regression

Inspired by recent advances in the field of expert-based approximations ...
research
02/02/2016

An analytic comparison of regularization methods for Gaussian Processes

Gaussian Processes (GPs) are a popular approach to predict the output of...

Please sign up or login with your details

Forgot password? Click here to reset