Training Support Vector Machines Using Frank-Wolfe Optimization Methods

12/04/2012
by   Emanuele Frandi, et al.
0

Training a Support Vector Machine (SVM) requires the solution of a quadratic programming problem (QP) whose computational complexity becomes prohibitively expensive for large scale datasets. Traditional optimization methods cannot be directly applied in these cases, mainly due to memory restrictions. By adopting a slightly different objective function and under mild conditions on the kernel used within the model, efficient algorithms to train SVMs have been devised under the name of Core Vector Machines (CVMs). This framework exploits the equivalence of the resulting learning problem with the task of building a Minimal Enclosing Ball (MEB) problem in a feature space, where data is implicitly embedded by a kernel function. In this paper, we improve on the CVM approach by proposing two novel methods to build SVMs based on the Frank-Wolfe algorithm, recently revisited as a fast method to approximate the solution of a MEB problem. In contrast to CVMs, our algorithms do not require to compute the solutions of a sequence of increasingly complex QPs and are defined by using only analytic optimization steps. Experiments on a large collection of datasets show that our methods scale better than CVMs in most cases, sometimes at the price of a slightly lower accuracy. As CVMs, the proposed methods can be easily extended to machine learning problems other than binary classification. However, effective classifiers are also obtained using kernels which do not satisfy the condition required by CVMs and can thus be used for a wider set of problems.

READ FULL TEXT

page 28

page 29

page 30

page 33

page 34

research
11/06/2020

Sequential Minimal Optimization for One-Class Slab Support Vector Machine

One Class Slab Support Vector Machines (OCSSVM) have turned out to be be...
research
12/17/2017

Super-sparse Learning in Similarity Spaces

In several applications, input samples are more naturally represented in...
research
10/26/2020

About the Non-Convex Optimization Problem Induced by Non-positive Semidefinite Kernel Learning

During the last years, kernel based methods proved to be very successful...
research
11/21/2016

An Efficient Training Algorithm for Kernel Survival Support Vector Machines

Survival analysis is a fundamental tool in medical research to identify ...
research
01/28/2019

Support Feature Machines

Support Vector Machines (SVMs) with various kernels have played dominant...
research
11/02/2011

Approximate Stochastic Subgradient Estimation Training for Support Vector Machines

Subgradient algorithms for training support vector machines have been qu...
research
04/03/2013

A Novel Frank-Wolfe Algorithm. Analysis and Applications to Large-Scale SVM Training

Recently, there has been a renewed interest in the machine learning comm...

Please sign up or login with your details

Forgot password? Click here to reset