Accelerated Linear SVM Training with Adaptive Variable Selection Frequencies

02/22/2013
by   Tobias Glasmachers, et al.
0

Support vector machine (SVM) training is an active research area since the dawn of the method. In recent years there has been increasing interest in specialized solvers for the important case of linear models. The algorithm presented by Hsieh et al., probably best known under the name of the "liblinear" implementation, marks a major breakthrough. The method is analog to established dual decomposition algorithms for training of non-linear SVMs, but with greatly reduced computational complexity per update step. This comes at the cost of not keeping track of the gradient of the objective any more, which excludes the application of highly developed working set selection algorithms. We present an algorithmic improvement to this method. We replace uniform working set selection with an online adaptation of selection frequencies. The adaptation criterion is inspired by modern second order working set selection methods. The same mechanism replaces the shrinking heuristic. This novel technique speeds up training in some cases by more than an order of magnitude.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/15/2014

Coordinate Descent with Online Adaptation of Coordinate Frequencies

Coordinate descent (CD) algorithms have become the method of choice for ...
research
06/19/2017

Modified Frank-Wolfe Algorithm for Enhanced Sparsity in Support Vector Machine Classifiers

This work proposes a new algorithm for training a re-weighted L2 Support...
research
10/02/2007

Structured variable selection in support vector machines

When applying the support vector machine (SVM) to high-dimensional class...
research
07/12/2011

BSVM: A Banded Suport Vector Machine

We describe a novel binary classification technique called Banded SVM (B...
research
12/19/2012

A complexity analysis of statistical learning algorithms

We apply information-based complexity analysis to support vector machine...
research
04/21/2020

A Mathematical Programming approach to Binary Supervised Classification with Label Noise

In this paper we propose novel methodologies to construct Support Vector...
research
08/10/2022

Classifier Transfer with Data Selection Strategies for Online Support Vector Machine Classification with Class Imbalance

Objective: Classifier transfers usually come with dataset shifts. To ove...

Please sign up or login with your details

Forgot password? Click here to reset