SVM via Saddle Point Optimization: New Bounds and Distributed Algorithms

05/20/2017
by   Yifei Jin, et al.
0

Support Vector Machine is one of the most classical approaches for classification and regression. Despite being studied for decades, obtaining practical algorithms for SVM is still an active research problem in machine learning. In this paper, we propose a new perspective for SVM via saddle point optimization. We provide an algorithm which achieves (1-ϵ)-approximations with running time Õ(nd+n√(d / ϵ)) for both separable (hard margin SVM) and non-separable cases (ν-SVM ), where n is the number of points and d is the dimensionality. To the best of our knowledge, the current best algorithm for hard margin SVM achieved by Gilbert algorithm gartner2009coresets requires O(nd / ϵ ) time. Our algorithm improves the running time by a factor of √(d)/√(ϵ). For ν-SVM, besides the well known quadratic programming approach which requires Ω(n^2 d) time joachims1998making,platt199912, no better algorithm is known. In the paper, we provide the first nearly linear time algorithm for ν-SVM. We also consider the distributed settings and provide distributed algorithms with low communication cost via saddle point optimization. Our algorithms require Õ(k(d +√(d/ϵ))) communication cost where k is the number of clients, almost matching the theoretical lower bound.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/19/2019

Quantum algorithms for Second-Order Cone Programming and Support Vector Machines

Second order cone programs (SOCPs) are a class of structured convex opti...
research
07/15/2023

A Nearly-Linear Time Algorithm for Structured Support Vector Machines

Quadratic programming is a fundamental problem in the field of convex op...
research
12/26/2020

Fast Training of Support Vector Machine for Forest Fire Prediction

Support Vector Machine (SVM) is a binary classification model, which aim...
research
03/05/2013

An Equivalence between the Lasso and Support Vector Machines

We investigate the relation of two fundamental tools in machine learning...
research
06/14/2020

Defending SVMs against Poisoning Attacks: the Hardness and DBSCAN Approach

Adversarial machine learning has attracted a great amount of attention i...
research
06/18/2020

Quantum algorithm for training nonlinear SVMs in almost linear time

We propose a quantum algorithm for training nonlinear support vector mac...
research
06/27/2012

The Greedy Miser: Learning under Test-time Budgets

As machine learning algorithms enter applications in industrial settings...

Please sign up or login with your details

Forgot password? Click here to reset