Estimating the Penalty Level of ℓ_1-minimization via Two Gaussian Approximation Methods

02/18/2020
by   Fang Xie, et al.
0

In this paper, we aim to give a theoretical approximation for the penalty level of ℓ_1-regularization problems. This can save much time in practice compared with the traditional methods, such as cross-validation. To achieve this goal, we develop two Gaussian approximation methods, which are based on a moderate deviation theorem and Stein's method respectively. Both of them give efficient approximations and have good performances in simulations. We apply the two Gaussian approximation methods into three types of ultra-high dimensional ℓ_1 penalized regressions: lasso, square-root lasso, and weighted ℓ_1 penalized Poisson regression. The numerical results indicate that our two ways to estimate the penalty levels achieve high computational efficiency. Besides, our prediction errors outperform that based on the 10-fold cross-validation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/10/2021

Analytic and Bootstrap-after-Cross-Validation Methods for Selecting Penalty Parameters of High-Dimensional M-Estimators

We develop two new methods for selecting the penalty parameter for the ℓ...
research
05/20/2020

On the use of cross-validation for the calibration of the tuning parameter in the adaptive lasso

The adaptive lasso is a popular extension of the lasso, which was shown ...
research
07/28/2014

Efficient Regularized Regression for Variable Selection with L0 Penalty

Variable (feature, gene, model, which we use interchangeably) selections...
research
03/10/2019

Lasso tuning through the flexible-weighted bootstrap

Regularized regression approaches such as the Lasso have been widely ado...
research
02/20/2019

Cross Validation for Penalized Quantile Regression with a Case-Weight Adjusted Solution Path

Cross validation is widely used for selecting tuning parameters in regul...
research
03/05/2023

Iterative Approximate Cross-Validation

Cross-validation (CV) is one of the most popular tools for assessing and...
research
08/23/2022

A flexible empirical Bayes approach to multiple linear regression and connections with penalized regression

We introduce a new empirical Bayes approach for large-scale multiple lin...

Please sign up or login with your details

Forgot password? Click here to reset