Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processes

04/27/2023
by   Louis C. Tiao, et al.
0

Despite their many desirable properties, Gaussian processes (GPs) are often compared unfavorably to deep neural networks (NNs) for lacking the ability to learn representations. Recent efforts to bridge the gap between GPs and deep NNs have yielded a new class of inter-domain variational GPs in which the inducing variables correspond to hidden units of a feedforward NN. In this work, we examine some practical issues associated with this approach and propose an extension that leverages the orthogonal decomposition of GPs to mitigate these limitations. In particular, we introduce spherical inter-domain features to construct more flexible data-dependent basis functions for both the principal and orthogonal components of the GP approximation and show that incorporating NN activation features under this framework not only alleviates these shortcomings but is more scalable than alternative strategies. Experiments on multiple benchmark datasets demonstrate the effectiveness of our approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/11/2023

Actually Sparse Variational Gaussian Processes

Gaussian processes (GPs) are typically criticised for their unfavourable...
research
06/30/2020

Sparse Gaussian Processes with Spherical Harmonic Features

We introduce a new class of inter-domain variational Gaussian processes ...
research
07/21/2021

Adaptive Inducing Points Selection For Gaussian Processes

Gaussian Processes (GPs) are flexible non-parametric models with strong ...
research
03/28/2023

Sparse Gaussian Processes with Spherical Harmonic Features Revisited

We revisit the Gaussian process model with spherical harmonic features a...
research
11/10/2020

Sparse within Sparse Gaussian Processes using Neighbor Information

Approximations to Gaussian processes based on inducing variables, combin...
research
02/19/2015

Scalable Bayesian Optimization Using Deep Neural Networks

Bayesian optimization is an effective methodology for the global optimiz...
research
06/07/2015

String Gaussian Process Kernels

We introduce a new class of nonstationary kernels, which we derive as co...

Please sign up or login with your details

Forgot password? Click here to reset