Penalized robust estimators in logistic regression with applications to sparse models

11/01/2019
by   Ana M. Bianco, et al.
0

Sparse covariates are frequent in classification and regression problems and in these settings the task of variable selection is usually of interest. As it is well known, sparse statistical models correspond to situations where there are only a small number of non–zero parameters and for that reason, they are much easier to interpret than dense ones. In this paper, we focus on the logistic regression model and our aim is to address robust and penalized estimation for the regression parameter. We introduce a family of penalized weighted M-type estimators for the logistic regression parameter that are stable against atypical data. We explore different penalizations functions and we introduce the so–called Sign penalization. This new penalty has the advantage that it depends only on one penalty parameter, avoiding arbitrary tuning constants. We discuss the variable selection capability of the given proposals as well as their asymptotic behaviour. Through a numerical study, we compare the finite sample performance of the proposal corresponding to different penalized estimators either robust or classical, under different scenarios. A robust cross–validation criterion is also presented. The analysis of two real data sets enables to investigate the stability of the penalized estimators to the presence of outliers.

READ FULL TEXT
research
01/28/2022

Asymptotic behaviour of penalized robust estimators in logistic regression when dimension increases

Penalized M-estimators for logistic regression models have been previous...
research
09/28/2011

Robust Parametric Classification and Variable Selection by a Minimum Distance Criterion

We investigate a robust penalized logistic regression algorithm based on...
research
10/01/2016

Tuning Parameter Calibration in High-dimensional Logistic Regression With Theoretical Guarantees

Feature selection is a standard approach to understanding and modeling h...
research
06/16/2020

Multi-Model Penalized Regression

Model fitting often aims to fit a single model, assuming that the impose...
research
03/28/2019

An analysis of the cost of hyper-parameter selection via split-sample validation, with applications to penalized regression

In the regression setting, given a set of hyper-parameters, a model-esti...
research
10/06/2020

Robust priors for regularized regression

Induction benefits from useful priors. Penalized regression approaches, ...
research
04/26/2023

A Statistical Interpretation of the Maximum Subarray Problem

Maximum subarray is a classical problem in computer science that given a...

Please sign up or login with your details

Forgot password? Click here to reset