Optimal Rates for Random Fourier Features

06/06/2015
by   Bharath K. Sriperumbudur, et al.
0

Kernel methods represent one of the most powerful tools in machine learning to tackle problems expressed in terms of function values and derivatives due to their capability to represent and model complex relations. While these methods show good versatility, they are computationally intensive and have poor scalability to large data as they require operations on Gram matrices. In order to mitigate this serious computational limitation, recently randomized constructions have been proposed in the literature, which allow the application of fast linear algorithms. Random Fourier features (RFF) are among the most popular and widely applied constructions: they provide an easily computable, low-dimensional feature representation for shift-invariant kernels. Despite the popularity of RFFs, very little is understood theoretically about their approximation quality. In this paper, we provide a detailed finite-sample theoretical analysis about the approximation quality of RFFs by (i) establishing optimal (in terms of the RFF dimension, and growing set size) performance guarantees in uniform norm, and (ii) presenting guarantees in L^r (1< r<∞) norms. We also propose an RFF approximation to derivatives of a kernel with a theoretical study on its approximation quality.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/11/2018

On Kernel Derivative Approximation with Random Fourier Features

Random Fourier features (RFF) represent one of the most popular and wide...
research
06/09/2015

On the Error of Random Fourier Features

Kernel methods give powerful, flexible, and theoretically grounded appro...
research
09/21/2011

Explicit Approximations of the Gaussian Kernel

We investigate training and using Gaussian kernel SVMs by approximating ...
research
10/01/2022

On The Relative Error of Random Fourier Features for Preserving Kernel Distance

The method of random Fourier features (RFF), proposed in a seminal paper...
research
06/20/2017

Statistical Consistency of Kernel PCA with Random Features

Kernel methods are powerful learning methodologies that provide a simple...
research
06/24/2018

A Unified Analysis of Random Fourier Features

We provide the first unified theoretical analysis of supervised learning...
research
03/03/2022

Uniform Approximations for Randomized Hadamard Transforms with Applications

Randomized Hadamard Transforms (RHTs) have emerged as a computationally ...

Please sign up or login with your details

Forgot password? Click here to reset