Compact Nonlinear Maps and Circulant Extensions

03/12/2015
by   Felix X. Yu, et al.
0

Kernel approximation via nonlinear random feature maps is widely used in speeding up kernel machines. There are two main challenges for the conventional kernel approximation methods. First, before performing kernel approximation, a good kernel has to be chosen. Picking a good kernel is a very challenging problem in itself. Second, high-dimensional maps are often required in order to achieve good performance. This leads to high computational cost in both generating the nonlinear maps, and in the subsequent learning and prediction process. In this work, we propose to optimize the nonlinear maps directly with respect to the classification objective in a data-dependent fashion. The proposed approach achieves kernel approximation and kernel learning in a joint framework. This leads to much more compact maps without hurting the performance. As a by-product, the same framework can also be used to achieve more compact kernel maps to approximate a known kernel. We also introduce Circulant Nonlinear Maps, which uses a circulant-structured projection matrix to speed up the nonlinear maps for high-dimensional data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/17/2013

Compact Random Feature Maps

Kernel approximation using randomized feature maps has recently gained a...
research
02/04/2016

Random Feature Maps via a Layered Random Projection (LaRP) Framework for Object Classification

The approximation of nonlinear kernels via linear feature maps has recen...
research
05/30/2018

On the Spectrum of Random Features Maps of High Dimensional Data

Random feature maps are ubiquitous in modern statistical machine learnin...
research
12/04/2020

Adaptive Explicit Kernel Minkowski Weighted K-means

The K-means algorithm is among the most commonly used data clustering me...
research
02/11/2018

Quadrature-based features for kernel approximation

We consider the problem of improving kernel approximation via randomized...
research
05/15/2021

Universality and Optimality of Structured Deep Kernel Networks

Kernel based methods yield approximation models that are flexible, effic...
research
09/23/2019

Scalable Kernel Learning via the Discriminant Information

Kernel approximation methods have been popular techniques for scalable k...

Please sign up or login with your details

Forgot password? Click here to reset