The distribution of the Lasso: Uniform control over sparse balls and adaptive parameter tuning

11/03/2018
by   Léo Miolane, et al.
0

The Lasso is a popular regression method for high-dimensional problems in which the number of parameters θ_1,...,θ_N, is larger than the number n of samples: N>n. A useful heuristics relates the statistical properties of the Lasso estimator to that of a simple soft-thresholding denoiser,in a denoising problem in which the parameters (θ_i)_i< N are observed in Gaussian noise, with a carefully tuned variance. Earlier work confirmed this picture in the limit n,N→∞, pointwise in the parameters θ, and in the value of the regularization parameter. Here, we consider a standard random design model and prove exponential concentration of its empirical distribution around the prediction provided by the Gaussian denoising model. Crucially, our results are uniform with respect to θ belonging to ℓ_q balls, q∈ [0,1], and with respect to the regularization parameter. This allows to derive sharp results for the performances of various data-driven procedures to tune the regularization. Our proofs make use of Gaussian comparison inequalities, and in particular of a version of Gordon's minimax theorem developed by Thrampoulidis, Oymak, and Hassibi, which controls the optimum value of the Lasso optimization problem. Crucially, we prove a stability property of the minimizer in Wasserstein distance, that allows to characterize properties of the minimizer itself.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/17/2013

Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory

We consider linear regression in the high-dimensional regime where the n...
research
03/18/2014

On the Sensitivity of the Lasso to the Number of Predictor Variables

The Lasso is a computationally efficient regression regularization proce...
research
01/30/2008

On the Distribution of the Adaptive LASSO Estimator

We study the distribution of the adaptive LASSO estimator (Zou (2006)) i...
research
07/27/2020

The Lasso with general Gaussian designs with applications to hypothesis testing

The Lasso is a method for high-dimensional regression, which is now comm...
research
04/24/2019

An Exploratory Analysis of Biased Learners in Soft-Sensing Frames

Data driven soft sensor design has recently gained immense popularity, d...
research
10/17/2020

On the best choice of Lasso program given data parameters

Generalized compressed sensing (GCS) is a paradigm in which a structured...
research
04/04/2018

The noise barrier and the large signal bias of the Lasso and other convex estimators

Convex estimators such as the Lasso, the matrix Lasso and the group Lass...

Please sign up or login with your details

Forgot password? Click here to reset