Hierarchically Compositional Kernels for Scalable Nonparametric Learning

08/02/2016
by   Jie Chen, et al.
0

We propose a novel class of kernels to alleviate the high computational cost of large-scale nonparametric learning with kernel methods. The proposed kernel is defined based on a hierarchical partitioning of the underlying data domain, where the Nyström method (a globally low-rank approximation) is married with a locally lossless approximation in a hierarchical fashion. The kernel maintains (strict) positive-definiteness. The corresponding kernel matrix admits a recursively off-diagonal low-rank structure, which allows for fast linear algebra computations. Suppressing the factor of data dimension, the memory and arithmetic complexities for training a regression or a classifier are reduced from O(n^2) and O(n^3) to O(nr) and O(nr^2), respectively, where n is the number of training examples and r is the rank on each level of the hierarchy. Although other randomized approximate kernels entail a similar complexity, empirical results show that the proposed kernel achieves a matching performance with a smaller r. We demonstrate comprehensive experiments to show the effective use of the proposed kernel on data sizes up to the order of millions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/05/2014

On the Complexity of Learning with Kernels

A well-recognized limitation of kernel learning is the requirement to ha...
research
05/03/2015

Structured Block Basis Factorization for Scalable Kernel Matrix Evaluation

Kernel matrices are popular in machine learning and scientific computing...
research
06/23/2017

On the numerical rank of radial basis function kernels in high dimension

Low-rank approximations are popular methods to reduce the high computati...
research
01/05/2021

Kernel optimization for Low-Rank Multi-Fidelity Algorithms

One of the major challenges for low-rank multi-fidelity (MF) approaches ...
research
06/04/2021

Kernel approximation on algebraic varieties

Low-rank approximation of kernels is a fundamental mathematical problem ...
research
02/26/2016

Multivariate Hawkes Processes for Large-scale Inference

In this paper, we present a framework for fitting multivariate Hawkes pr...
research
10/12/2017

New efficient algorithms for multiple change-point detection with kernels

Several statistical approaches based on reproducing kernels have been pr...

Please sign up or login with your details

Forgot password? Click here to reset