Stochastic Variational Deep Kernel Learning

11/01/2016
by   Andrew Gordon Wilson, et al.
0

Deep kernel learning combines the non-parametric flexibility of kernel methods with the inductive biases of deep learning architectures. We propose a novel deep kernel learning model and stochastic variational inference procedure which generalizes deep kernel learning approaches to enable classification, multi-task learning, additive covariance structures, and stochastic gradient training. Specifically, we apply additive base kernels to subsets of output features from deep neural architectures, and jointly learn the parameters of the base kernels and deep network through a Gaussian process marginal likelihood objective. Within this framework, we derive an efficient form of stochastic variational inference which leverages local kernel interpolation, inducing points, and structure exploiting algebra. We show improved performance over stand alone deep networks, SVMs, and state of the art scalable Gaussian processes on several classification benchmarks, including an airline delay dataset containing 6 million training points, CIFAR, and ImageNet.

READ FULL TEXT
research
11/06/2015

Deep Kernel Learning

We introduce scalable deep kernels, which combine the structural propert...
research
10/19/2017

Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition

We propose a method (TT-GP) for approximate inference in Gaussian Proces...
research
10/27/2016

Learning Scalable Deep Kernels with Recurrent Structure

Many applications in speech, robotics, finance, and biology deal with se...
research
09/08/2023

Parallel and Limited Data Voice Conversion Using Stochastic Variational Deep Kernel Learning

Typically, voice conversion is regarded as an engineering problem with l...
research
08/13/2018

Kernel Flows: from learning kernels from data into the abyss

Learning can be seen as approximating an unknown function by interpolati...
research
10/04/2020

Deep kernel processes

We define deep kernel processes in which positive definite Gram matrices...
research
10/18/2020

Movement-induced Priors for Deep Stereo

We propose a method for fusing stereo disparity estimation with movement...

Please sign up or login with your details

Forgot password? Click here to reset