Estimating the Penalty Level of ℓ_1-minimization via Two Gaussian Approximation Methods
share
In this paper, we aim to give a theoretical approximation for the penalty level of ℓ_1-regularization problems. This can save much time in practice compared with the traditional methods, such as cross-validation. To achieve this goal, we develop two Gaussian approximation methods, which are based on a moderate deviation theorem and Stein's method respectively. Both of them give efficient approximations and have good performances in simulations. We apply the two Gaussian approximation methods into three types of ultra-high dimensional ℓ_1 penalized regressions: lasso, square-root lasso, and weighted ℓ_1 penalized Poisson regression. The numerical results indicate that our two ways to estimate the penalty levels achieve high computational efficiency. Besides, our prediction errors outperform that based on the 10-fold cross-validation.
READ FULL TEXT