Behavior of Lasso and Lasso-based inference under limited variability

03/20/2019
by   Kaspar Wuthrich, et al.
0

We study the nonasymptotic behavior of Lasso and Lasso-based inference when the covariates exhibit limited variability, which does not appear to have been considered in the literature, despite its prevalence in applied research. In settings that are generally considered favorable to Lasso, we show that, if the absolute value of a nonzero regression coefficient is smaller or equal to a threshold, Lasso fails to select the corresponding covariate with high probability (approaching to 1 asymptotically). In particular, limited variability can render Lasso unable to select even those covariates with coefficients that are well-separated from zero. Moreover, based on simple theoretical examples, we show that post double Lasso and debiased Lasso can exhibit size distortions under limited variability. Monte Carlo simulations corroborate our theoretical results and further demonstrate that, under limited variability, the performance of Lasso and Lasso-based inference methods is very sensitive to the choice of the penalty parameter. This begs the question of how to make statistical inference (e.g., constructing confidence intervals) under limited variability. In moderately high-dimensional problems, where the number of covariates is large but still smaller than the sample size, OLS constitutes a natural alternative to Lasso-based inference methods. In empirically relevant settings, our simulation results show that, under limited variability, OLS with recently developed standard errors, which are proven robust to many covariates, demonstrates a superior finite sample performance relative to Lasso-based inference methods.

READ FULL TEXT
research
03/20/2019

Omitted variable bias of Lasso-based inference methods under limited variability: A finite sample analysis

We study the finite sample behavior of Lasso and Lasso-based inference m...
research
11/09/2017

Debiasing the Debiased Lasso with Bootstrap

In this paper, we prove that under proper conditions, bootstrap can furt...
research
07/14/2022

High Dimensional Generalised Penalised Least Squares

In this paper we develop inference for high dimensional linear models, w...
research
08/05/2008

Support union recovery in high-dimensional multivariate regression

In multivariate regression, a K-dimensional response vector is regressed...
research
11/04/2019

Online Debiasing for Adaptively Collected High-dimensional Data

Adaptive collection of data is increasingly commonplace in many applicat...
research
02/07/2014

On the Prediction Performance of the Lasso

Although the Lasso has been extensively studied, the relationship betwee...
research
06/14/2021

Robust Inference for High-Dimensional Linear Models via Residual Randomization

We propose a residual randomization procedure designed for robust Lasso-...

Please sign up or login with your details

Forgot password? Click here to reset