DeepAI AI Chat
Log In Sign Up

Generalization for multiclass classification with overparameterized linear models

06/03/2022
by   Vignesh Subramanian, et al.
0

Via an overparameterized linear model with Gaussian features, we provide conditions for good generalization for multiclass classification of minimum-norm interpolating solutions in an asymptotic setting where both the number of underlying features and the number of classes scale with the number of training points. The survival/contamination analysis framework for understanding the behavior of overparameterized learning problems is adapted to this setting, revealing that multiclass classification qualitatively behaves like binary classification in that, as long as there are not too many classes (made precise in the paper), it is possible to generalize well even in some settings where the corresponding regression tasks would not generalize. Besides various technical challenges, it turns out that the key difference from the binary classification setting is that there are relatively fewer positive training examples of each class in the multiclass setting as the number of classes increases, making the multiclass problem "harder" than the binary one.

READ FULL TEXT

page 7

page 11

11/16/2020

Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View

Contemporary machine learning applications often involve classification ...
09/06/2012

Multiclass Learning with Simplex Coding

In this paper we discuss a novel framework for multiclass learning, defi...
10/04/2018

Improved generalization bounds for robust learning

We consider a model of robust learning in an adversarial environment. Th...
02/18/2023

The Generalization Error of Stochastic Mirror Descent on Over-Parametrized Linear Models

Despite being highly over-parametrized, and having the ability to fully ...
06/15/2016

Logarithmic Time One-Against-Some

We create a new online reduction of multiclass classification to binary ...
05/01/2020

Generalization Error of Generalized Linear Models in High Dimensions

At the heart of machine learning lies the question of generalizability o...
02/06/2020

Quantification of Differential Information using Matrix Pencil

Any traditional classification problem in general involves modelling ind...