DeepAI AI Chat
Log In Sign Up

Learning primal-dual sparse kernel machines

by   Riikka Huusari, et al.

Traditionally, kernel methods rely on the representer theorem which states that the solution to a learning problem is obtained as a linear combination of the data mapped into the reproducing kernel Hilbert space (RKHS). While elegant from theoretical point of view, the theorem is prohibitive for algorithms' scalability to large datasets, and the interpretability of the learned function. In this paper, instead of using the traditional representer theorem, we propose to search for a solution in RKHS that has a pre-image decomposition in the original data space, where the elements don't necessarily correspond to the elements in the training set. Our gradient-based optimisation method then hinges on optimising over possibly sparse elements in the input space, and enables us to obtain a kernel-based model with both primal and dual sparsity. We give theoretical justification on the proposed method's generalization ability via a Rademacher bound. Our experiments demonstrate a better scalability and interpretability with accuracy on par with the traditional kernel-based models.


Margins, Kernels and Non-linear Smoothed Perceptrons

We focus on the problem of finding a non-linear classification function ...

Duality for Neural Networks through Reproducing Kernel Banach Spaces

Reproducing Kernel Hilbert spaces (RKHS) have been a very successful too...

Sparse Kernel Canonical Correlation Analysis via ℓ_1-regularization

Canonical correlation analysis (CCA) is a multivariate statistical techn...

Functional learning through kernels

This paper reviews the functional aspects of statistical learning theory...

How to improve the interpretability of kernel learning

In recent years, machine learning researchers have focused on methods to...

Bayesian learning of feature spaces for multitasks problems

This paper presents a Bayesian framework to construct non-linear, parsim...

Robust One-Class Kernel Spectral Regression

The kernel null-space technique and its regression-based formulation (ca...