On the Error Resistance of Hinge Loss Minimization

12/02/2020
by   Kunal Talwar, et al.
0

Commonly used classification algorithms in machine learning, such as support vector machines, minimize a convex surrogate loss on training examples. In practice, these algorithms are surprisingly robust to errors in the training data. In this work, we identify a set of conditions on the data under which such surrogate loss minimization algorithms provably learn the correct classifier. This allows us to establish, in a unified framework, the robustness of these algorithms under various models on data as well as error. In particular, we show that if the data is linearly classifiable with a slightly non-trivial margin (i.e. a margin at least C/√(d) for d-dimensional unit vectors), and the class-conditional distributions are near isotropic and logconcave, then surrogate loss minimization has negligible error on the uncorrupted data even when a constant fraction of examples are adversarially mislabeled.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/18/2018

Consistent Robust Adversarial Prediction for General Multiclass Classification

We propose a robust adversarial prediction framework for general multicl...
research
06/24/2021

Constrained Classification and Policy Learning

Modern machine learning approaches to classification, including AdaBoost...
research
09/22/2020

On the proliferation of support vectors in high dimensions

The support vector machine (SVM) is a well-established classification me...
research
05/28/2020

Calibrated Surrogate Losses for Adversarially Robust Classification

Adversarially robust classification seeks a classifier that is insensiti...
research
12/20/2017

Adversarial Structured Prediction for Multivariate Measures

Many predicted structured objects (e.g., sequences, matchings, trees) ar...
research
10/14/2021

Towards Understanding the Data Dependency of Mixup-style Training

In the Mixup training paradigm, a model is trained using convex combinat...
research
03/10/2018

A Minimax Surrogate Loss Approach to Conditional Difference Estimation

We present a new machine learning approach to estimate personalized trea...

Please sign up or login with your details

Forgot password? Click here to reset