Fast Sketching of Polynomial Kernels of Polynomial Degree

08/21/2021
by   Zhao Song, et al.
0

Kernel methods are fundamental in machine learning, and faster algorithms for kernel approximation provide direct speedups for many core tasks in machine learning. The polynomial kernel is especially important as other kernels can often be approximated by the polynomial kernel via a Taylor series expansion. Recent techniques in oblivious sketching reduce the dependence in the running time on the degree q of the polynomial kernel from exponential to polynomial, which is useful for the Gaussian kernel, for which q can be chosen to be polylogarithmic. However, for more slowly growing kernels, such as the neural tangent and arc-cosine kernels, q needs to be polynomial, and previous work incurs a polynomial factor slowdown in the running time. We give a new oblivious sketch which greatly improves upon this running time, by removing the dependence on q in the leading order term. Combined with a novel sampling scheme, we give the fastest algorithms for approximating a large family of slow-growing kernels.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/03/2019

Oblivious Sketching of High-Degree Polynomial Kernels

Kernel methods are fundamental tools in machine learning that allow dete...
research
12/22/2017

Learning the Kernel for Classification and Regression

We investigate a series of learning kernel problems with polynomial comb...
research
02/09/2022

Leverage Score Sampling for Tensor Product Matrices in Input Sparsity Time

We give an input sparsity time sampling algorithm for spectrally approxi...
research
05/12/2022

Optimal-Degree Polynomial Approximations for Exponentials and Gaussian Kernel Density Estimation

For any real numbers B ≥ 1 and δ∈ (0, 1) and function f: [0, B] →ℝ, let ...
research
11/02/2014

Fast Randomized Kernel Methods With Statistical Guarantees

One approach to improving the running time of kernel-based machine learn...
research
06/15/2021

Scaling Neural Tangent Kernels via Sketching and Random Features

The Neural Tangent Kernel (NTK) characterizes the behavior of infinitely...
research
09/21/2011

Explicit Approximations of the Gaussian Kernel

We investigate training and using Gaussian kernel SVMs by approximating ...

Please sign up or login with your details

Forgot password? Click here to reset