DeepAI AI Chat
Log In Sign Up

Generalization for multiclass classification with overparameterized linear models

by   Vignesh Subramanian, et al.

Via an overparameterized linear model with Gaussian features, we provide conditions for good generalization for multiclass classification of minimum-norm interpolating solutions in an asymptotic setting where both the number of underlying features and the number of classes scale with the number of training points. The survival/contamination analysis framework for understanding the behavior of overparameterized learning problems is adapted to this setting, revealing that multiclass classification qualitatively behaves like binary classification in that, as long as there are not too many classes (made precise in the paper), it is possible to generalize well even in some settings where the corresponding regression tasks would not generalize. Besides various technical challenges, it turns out that the key difference from the binary classification setting is that there are relatively fewer positive training examples of each class in the multiclass setting as the number of classes increases, making the multiclass problem "harder" than the binary one.


page 7

page 11


Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View

Contemporary machine learning applications often involve classification ...

Multiclass Learning with Simplex Coding

In this paper we discuss a novel framework for multiclass learning, defi...

Improved generalization bounds for robust learning

We consider a model of robust learning in an adversarial environment. Th...

The Generalization Error of Stochastic Mirror Descent on Over-Parametrized Linear Models

Despite being highly over-parametrized, and having the ability to fully ...

Logarithmic Time One-Against-Some

We create a new online reduction of multiclass classification to binary ...

Generalization Error of Generalized Linear Models in High Dimensions

At the heart of machine learning lies the question of generalizability o...

Quantification of Differential Information using Matrix Pencil

Any traditional classification problem in general involves modelling ind...