Similarity Learning for Provably Accurate Sparse Linear Classification

06/27/2012
by   Aurélien Bellet, et al.
0

In recent years, the crucial importance of metrics in machine learning algorithms has led to an increasing interest for optimizing distance and similarity functions. Most of the state of the art focus on learning Mahalanobis distances (requiring to fulfill a constraint of positive semi-definiteness) for use in a local k-NN algorithm. However, no theoretical link is established between the learned metrics and their performance in classification. In this paper, we make use of the formal framework of good similarities introduced by Balcan et al. to design an algorithm for learning a non PSD linear similarity optimized in a nonlinear feature space, which is then used to build a global linear classifier. We show that our approach has uniform stability and derive a generalization bound on the classification error. Experiments performed on various datasets confirm the effectiveness of our approach compared to state-of-the-art methods and provide evidence that (i) it is fast, (ii) robust to overfitting and (iii) produces very sparse classifiers.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/17/2013

Supervised Metric Learning with Generalization Guarantees

The crucial importance of metrics in machine learning algorithms has led...
research
01/22/2010

Classifying Network Data with Deep Kernel Machines

Inspired by a growing interest in analyzing network data, we study the p...
research
04/07/2018

Not quite unreasonable effectiveness of machine learning algorithms

State-of-the-art machine learning algorithms demonstrate close to absolu...
research
06/21/2019

On Tree-based Methods for Similarity Learning

In many situations, the choice of an adequate similarity measure or metr...
research
04/15/2021

Sparse online relative similarity learning

For many data mining and machine learning tasks, the quality of a simila...
research
12/22/2011

Similarity-based Learning via Data Driven Embeddings

We consider the problem of classification using similarity/distance func...
research
05/13/2023

A Novel Memetic Strategy for Optimized Learning of Classification Trees

Given the increasing interest in interpretable machine learning, classif...

Please sign up or login with your details

Forgot password? Click here to reset