DeepAI AI Chat
Log In Sign Up

Variational Inference for Gaussian Process Models with Linear Complexity

11/28/2017
by   Ching-An Cheng, et al.
Georgia Institute of Technology
0

Large-scale Gaussian process inference has long faced practical challenges due to time and space complexity that is superlinear in dataset size. While sparse variational Gaussian process models are capable of learning from large-scale data, standard strategies for sparsifying the model can prevent the approximation of complex functions. In this work, we propose a novel variational Gaussian process model that decouples the representation of mean and covariance functions in reproducing kernel Hilbert space. We show that this new parametrization generalizes previous models. Furthermore, it yields a variational inference problem that can be solved by stochastic gradient ascent with time and space complexity that is only linear in the number of mean function parameters, regardless of the choice of kernels, likelihoods, and inducing points. This strategy makes the adoption of large-scale expressive Gaussian process models possible. We run several experiments on regression tasks and show that this decoupled approach greatly outperforms previous sparse variational Gaussian process inference procedures.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/10/2015

Stochastic Expectation Propagation for Large Scale Gaussian Process Classification

A method for large scale Gaussian process classification has been recent...
10/24/2011

Multiple Gaussian Process Models

We consider a Gaussian process formulation of the multiple kernel learni...
02/27/2017

Semi-parametric Network Structure Discovery Models

We propose a network structure discovery model for continuous observatio...
03/09/2015

Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs

Standard sparse pseudo-input approximations to the Gaussian process (GP)...
07/14/2021

Spectrum Gaussian Processes Based On Tunable Basis Functions

Spectral approximation and variational inducing learning for the Gaussia...
09/08/2023

Parallel and Limited Data Voice Conversion Using Stochastic Variational Deep Kernel Learning

Typically, voice conversion is regarded as an engineering problem with l...
02/11/2020

Large Scale Tensor Regression using Kernels and Variational Inference

We outline an inherent weakness of tensor factorization models when late...