Asymptotic equivalence of regularization methods in thresholded parameter space

05/11/2016
by   Yingying Fan, et al.
0

High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this paper, we characterize the asymptotic equivalence of regularization methods, with general penalty functions, in a thresholded parameter space under the generalized linear model setting, where the dimensionality can grow up to exponentially with the sample size. To assess their performance, we establish the oracle inequalities, as in Bickel, Ritov and Tsybakov (2009), of the global minimizer for these methods under various prediction and variable selection losses. These results reveal an interesting phase transition phenomenon. For polynomially growing dimensionality, the L_1-regularization method of Lasso and concave methods are asymptotically equivalent, having the same convergence rates in the oracle inequalities. For exponentially growing dimensionality, concave methods are asymptotically equivalent but have faster convergence rates than the Lasso. We also establish a stronger property of the oracle risk inequalities of the regularization methods, as well as the sampling properties of computable solutions. Our new theoretical results are illustrated and justified by simulation and real data examples.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/11/2016

Asymptotic properties for combined L_1 and concave regularization

Two important goals of high-dimensional modeling are prediction and vari...
research
05/11/2016

High dimensional thresholded regression and shrinkage effect

High-dimensional sparse modeling via regularization provides a powerful ...
research
09/14/2019

Higher Order Refinements by Bootstrap in Lasso and other Penalized Regression Methods

Selection of important covariates and to drop the unimportant ones from ...
research
05/11/2016

The constrained Dantzig selector with enhanced consistency

The Dantzig selector has received popularity for many applications such ...
research
06/27/2013

Optimal Feature Selection in High-Dimensional Discriminant Analysis

We consider the high-dimensional discriminant analysis problem. For this...
research
10/13/2017

Enumerating Multiple Equivalent Lasso Solutions

Predictive modelling is a data-analysis task common in many scientific f...
research
06/01/2023

From sparse to dense functional data in high dimensions: Revisiting phase transitions from a non-asymptotic perspective

Nonparametric estimation of the mean and covariance functions is ubiquit...

Please sign up or login with your details

Forgot password? Click here to reset