Distance Metric Learning for Kernel Machines

08/16/2012
by   Zhixiang Xu, et al.
0

Recent work in metric learning has significantly improved the state-of-the-art in k-nearest neighbor classification. Support vector machines (SVM), particularly with RBF kernels, are amongst the most popular classification algorithms that uses distance metrics to compare examples. This paper provides an empirical analysis of the efficacy of three of the most popular Mahalanobis metric learning algorithms as pre-processing for SVM training. We show that none of these algorithms generate metrics that lead to particularly satisfying improvements for SVM-RBF classification. As a remedy we introduce support vector metric learning (SVML), a novel algorithm that seamlessly combines the learning of a Mahalanobis metric with the training of the RBF-SVM parameters. We demonstrate the capabilities of SVML on nine benchmark data sets of varying sizes and difficulties. In our study, SVML outperforms all alternative state-of-the-art metric learning algorithms in terms of accuracy and establishes itself as a serious alternative to the standard Euclidean metric with model selection by cross validation.

READ FULL TEXT
research
02/02/2015

Iterated Support Vector Machines for Distance Metric Learning

Distance metric learning aims to learn from the given training data a va...
research
01/23/2012

A metric learning perspective of SVM: on the relation of SVM and LMNN

Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor alg...
research
08/13/2019

metric-learn: Metric Learning Algorithms in Python

metric-learn is an open source Python package implementing supervised an...
research
03/11/2016

Nonstationary Distance Metric Learning

Recent work in distance metric learning has focused on learning transfor...
research
06/01/2021

Weighting vectors for machine learning: numerical harmonic analysis applied to boundary detection

Metric space magnitude, an active field of research in algebraic topolog...
research
08/16/2021

Evolutionary competitive swarm exploring optimal support vector machines and feature weighting

To deal with classification problems, support vector machines (SVMs) are...
research
12/25/2017

Kernel Regression with Sparse Metric Learning

Kernel regression is a popular non-parametric fitting technique. It aims...

Please sign up or login with your details

Forgot password? Click here to reset