Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures

04/28/2021
by   Yuan Cao, et al.
4

Modern machine learning systems such as deep neural networks are often highly over-parameterized so that they can fit the noisy training data exactly, yet they can still achieve small test errors in practice. In this paper, we study this "benign overfitting" (Bartlett et al. (2020)) phenomenon of the maximum margin classifier for linear classification problems. Specifically, we consider data generated from sub-Gaussian mixtures, and provide a tight risk bound for the maximum margin linear classifier in the over-parameterized setting. Our results precisely characterize the condition under which benign overfitting can occur in linear classification problems, and improve on previous work. They also have direct implications for over-parameterized logistic regression.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2021

Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation

The growing literature on "benign overfitting" in overparameterized mode...
research
12/31/2021

Benign Overfitting in Adversarially Robust Linear Classification

"Benign overfitting", where classifiers memorize noisy training data yet...
research
06/07/2021

Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensions

Generalised linear models for multi-class classification problems are on...
research
11/18/2020

Benign Overfitting in Binary Classification of Gaussian Mixtures

Deep neural networks generalize well despite being exceedingly overparam...
research
01/27/2022

The Implicit Bias of Benign Overfitting

The phenomenon of benign overfitting, where a predictor perfectly fits n...
research
04/25/2020

Finite-sample analysis of interpolating linear classifiers in the overparameterized regime

We prove bounds on the population risk of the maximum margin algorithm f...
research
06/11/2020

A new measure for overfitting and its implications for backdooring of deep learning

Overfitting describes the phenomenon that a machine learning model fits ...

Please sign up or login with your details

Forgot password? Click here to reset