The Fast Kernel Transform

06/08/2021
by   John Paul Ryan, et al.
10

Kernel methods are a highly effective and widely used collection of modern machine learning algorithms. A fundamental limitation of virtually all such methods are computations involving the kernel matrix that naively scale quadratically (e.g., constructing the kernel matrix and matrix-vector multiplication) or cubically (solving linear systems) with the size of the data set N. We propose the Fast Kernel Transform (FKT), a general algorithm to compute matrix-vector multiplications (MVMs) for datasets in moderate dimensions with quasilinear complexity. Typically, analytically grounded fast multiplication methods require specialized development for specific kernels. In contrast, our scheme is based on auto-differentiation and automated symbolic computations that leverage the analytical structure of the underlying kernel. This allows the FKT to be easily applied to a broad class of kernels, including Gaussian, Matern, and Rational Quadratic covariance functions and physically motivated Green's functions, including those of the Laplace and Helmholtz equations. Furthermore, the FKT maintains a high, quantifiable, and controllable level of accuracy – properties that many acceleration methods lack. We illustrate the efficacy and versatility of the FKT by providing timing and accuracy benchmarks and by applying it to scale the stochastic neighborhood embedding (t-SNE) and Gaussian processes to large real-world data sets.

READ FULL TEXT
research
11/19/2021

Learning in High-Dimensional Feature Spaces Using ANOVA-Based Fast Matrix-Vector Multiplication

Kernel matrices are crucial in many learning tasks such as support vecto...
research
02/24/2022

A Dynamic Fast Gaussian Transform

The Fast Gaussian Transform (FGT) enables subquadratic-time multiplicati...
research
06/19/2020

Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization

Matrix square roots and their inverses arise frequently in machine learn...
research
02/16/2021

Faster Kernel Matrix Algebra via Density Estimation

We study fast algorithms for computing fundamental properties of a posit...
research
02/02/2022

Giga-scale Kernel Matrix Vector Multiplication on GPU

Kernel matrix-vector multiplication (KMVM) is a foundational operation i...
research
03/06/2019

PBBFMM3D: a Parallel Black-Box Fast Multipole Method for Non-oscillatory Kernels

This paper presents PBBFMM3D: a parallel black-box fast multipole method...
research
12/06/2007

Kernels and Ensembles: Perspectives on Statistical Learning

Since their emergence in the 1990's, the support vector machine and the ...

Please sign up or login with your details

Forgot password? Click here to reset