Bolasso: model consistent Lasso estimation through the bootstrap

04/08/2008
by   Francis Bach, et al.
0

We consider the least-square linear regression problem with regularization by the l1-norm, a problem usually referred to as the Lasso. In this paper, we present a detailed asymptotic analysis of model consistency of the Lasso. For various decays of the regularization parameter, we compute asymptotic equivalents of the probability of correct model selection (i.e., variable selection). For a specific rate decay, we show that the Lasso selects all the variables that should enter the model with probability tending to one exponentially fast, while it selects all other variables with strictly positive probability. We show that this property implies that if we run the Lasso for several bootstrapped replications of a given sample, then intersecting the supports of the Lasso bootstrap estimates leads to consistent model selection. This novel variable selection algorithm, referred to as the Bolasso, is compared favorably to other linear regression methods on synthetic data and datasets from the UCI machine learning repository.

READ FULL TEXT

page 20

page 21

page 22

page 26

research
11/05/2010

The Loss Rank Criterion for Variable Selection in Linear Regression Analysis

Lasso and other regularization procedures are attractive methods for var...
research
09/14/2019

Higher Order Refinements by Bootstrap in Lasso and other Penalized Regression Methods

Selection of important covariates and to drop the unimportant ones from ...
research
06/19/2018

Simultaneous Signal Subspace Rank and Model Selection with an Application to Single-snapshot Source Localization

This paper proposes a novel method for model selection in linear regress...
research
03/27/2019

Asymptotics and Optimal Designs of SLOPE for Sparse Linear Regression

In sparse linear regression, the SLOPE estimator generalizes LASSO by as...
research
05/07/2023

Provable Identifiability of Two-Layer ReLU Neural Networks via LASSO Regularization

LASSO regularization is a popular regression tool to enhance the predict...
research
08/30/2023

Adaptive Lasso, Transfer Lasso, and Beyond: An Asymptotic Perspective

This paper presents a comprehensive exploration of the theoretical prope...
research
02/28/2018

Semi-Analytic Resampling in Lasso

An approximate method for conducting resampling in Lasso, the ℓ_1 penali...

Please sign up or login with your details

Forgot password? Click here to reset