Learning Data-adaptive Nonparametric Kernels

08/31/2018
by   Fanghui Liu, et al.
8

Traditional kernels or their combinations are often not sufficiently flexible to fit the data in complicated practical tasks. In this paper, we present a Data-Adaptive Nonparametric Kernel (DANK) learning framework by imposing an adaptive matrix on the kernel/Gram matrix in an entry-wise strategy. Since we do not specify the formulation of the adaptive matrix, each entry in it can be directly and flexibly learned from the data. Therefore, the solution space of the learned kernel is largely expanded, which makes DANK flexible to adapt to the data. Specifically, the proposed kernel learning framework can be seamlessly embedded to support vector machines (SVM) and support vector regression (SVR), which has the capability of enlarging the margin between classes and reducing the model generalization error. Theoretically, we demonstrate that the objective function of our devised model is gradient-Lipschitz continuous. Thereby, the training process for kernel and parameter learning in SVM/SVR can be efficiently optimized in a unified framework. Further, to address the scalability issue in DANK, a decomposition-based scalable approach is developed, of which the effectiveness is demonstrated by both empirical studies and theoretical guarantees. Experimentally, our method outperforms other representative kernel learning based algorithms on various classification and regression benchmark datasets.

READ FULL TEXT
research
04/17/2023

Snacks: a fast large-scale kernel SVM solver

Kernel methods provide a powerful framework for non parametric learning....
research
03/09/2013

Complex Support Vector Machines for Regression and Quaternary Classification

The paper presents a new framework for complex Support Vector Regression...
research
11/05/2020

AML-SVM: Adaptive Multilevel Learning with Support Vector Machines

The support vector machines (SVM) is one of the most widely used and pra...
research
11/08/2019

Adaptive Kernel Value Caching for SVM Training

Support Vector Machines (SVMs) can solve structured multi-output learnin...
research
02/27/2019

TrIK-SVM : an alternative decomposition for kernel methods in Krein spaces

The proposed work aims at proposing a alternative kernel decomposition i...
research
09/26/2018

Generalization Properties of hyper-RKHS and its Application to Out-of-Sample Extensions

Hyper-kernels endowed by hyper-Reproducing Kernel Hilbert Space (hyper-R...
research
02/29/2012

Learning from Distributions via Support Measure Machines

This paper presents a kernel-based discriminative learning framework on ...

Please sign up or login with your details

Forgot password? Click here to reset