Multiclass classification by sparse multinomial logistic regression

03/04/2020
by   Felix Abramovich, et al.
0

In this paper we consider high-dimensional multiclass classification by sparse multinomial logistic regression extending the results of Abramovich and Grinshtein (2019) for the binary case. We propose a feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the nonasymptotic bounds for misclassification excess risk of the resulting classifier. We establish also their tightness by deriving the corresponding minimax lower bounds. In particular, we show that there exist two regimes corresponding to small and large number of classes. The bounds can be reduced under the additional low noise condition. Implementation of any complexity penalty based procedure, however, requires a combinatorial search over all possible models. To find a feature selection procedure computationally feasible for high-dimensional data, we propose multinomial logistic group Lasso and Slope classifiers and show that they also achieve the optimal order in the minimax sense.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/26/2017

High-dimensional classification by sparse logistic regression

We consider high-dimensional binary classification by sparse logistic re...
research
04/13/2022

Generalization Error Bounds for Multiclass Sparse Linear Classifiers

We consider high-dimensional multiclass classification by sparse multino...
research
03/01/2021

Adaptive Sampling for Minimax Fair Classification

Machine learning models trained on imbalanced datasets can often end up ...
research
10/01/2016

Tuning Parameter Calibration in High-dimensional Logistic Regression With Theoretical Guarantees

Feature selection is a standard approach to understanding and modeling h...
research
05/23/2019

Naive Feature Selection: Sparsity in Naive Bayes

Due to its linear complexity, naive Bayes classification remains an attr...
research
03/19/2018

Optimal link prediction with matrix logistic regression

We consider the problem of link prediction, based on partial observation...
research
10/03/2019

Minimax Bounds for Distributed Logistic Regression

We consider a distributed logistic regression problem where labeled data...

Please sign up or login with your details

Forgot password? Click here to reset