Deep Kernel Learning via Random Fourier Features

10/07/2019
by   Jiaxuan Xie, et al.
0

Kernel learning methods are among the most effective learning methods and have been vigorously studied in the past decades. However, when tackling with complicated tasks, classical kernel methods are not flexible or "rich" enough to describe the data and hence could not yield satisfactory performance. In this paper, via Random Fourier Features (RFF), we successfully incorporate the deep architecture into kernel learning, which significantly boosts the flexibility and richness of kernel machines while keeps kernels' advantage of pairwise handling small data. With RFF, we could establish a deep structure and make every kernel in RFF layers could be trained end-to-end. Since RFF with different distributions could represent different kernels, our model has the capability of finding suitable kernels for each layer, which is much more flexible than traditional kernel-based methods where the kernel is pre-selected. This fact also helps yield a more sophisticated kernel cascade connection in the architecture. On small datasets (less than 1000 samples), for which deep learning is generally not suitable due to overfitting, our method achieves superior performance compared to advanced kernel methods. On large-scale datasets, including non-image and image classification tasks, our method also has competitive performance.

READ FULL TEXT
research
11/27/2017

Data Dependent Kernel Approximation using Pseudo Random Fourier Features

Kernel methods are powerful and flexible approach to solve many problems...
research
04/16/2018

Deep Embedding Kernel

In this paper, we propose a novel supervised learning method that is cal...
research
09/10/2020

End-to-end Kernel Learning via Generative Random Fourier Features

Random Fourier features enable researchers to build feature map to learn...
research
09/07/2022

Bayesian learning of feature spaces for multitasks problems

This paper presents a Bayesian framework to construct non-linear, parsim...
research
12/19/2014

A la Carte - Learning Fast Kernels

Kernel methods have great promise for learning rich statistical represen...
research
05/10/2018

Supervising Nyström Methods via Negative Margin Support Vector Selection

Pattern recognition on big data can be challenging for kernel machines a...
research
02/26/2019

Implicit Kernel Learning

Kernels are powerful and versatile tools in machine learning and statist...

Please sign up or login with your details

Forgot password? Click here to reset