Variable Selection in High Dimensions with Random Designs and Orthogonal Matching Pursuit

09/04/2011
by   Antony Joseph, et al.
0

The performance of Orthogonal Matching Pursuit (OMP) for variable selection is analyzed for random designs. When contrasted with the deterministic case, since the performance is here measured after averaging over the distribution of the design matrix, one can have far less stringent sparsity constraints on the coefficient vector. We demonstrate that for exact sparse vectors, the performance of the OMP is similar to known results on the Lasso algorithm [IEEE Trans. Inform. Theory55 (2009) 2183--2202]. Moreover, variable selection under a more relaxed sparsity assumption on the coefficient vector, whereby one has only control on the ℓ_1 norm of the smaller coefficients, is also analyzed. As a consequence of these results, we also show that the coefficient estimate satisfies strong oracle type inequalities.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/07/2021

Bounded support in linear random coefficient models: Identification and variable selection

We consider linear random coefficient regression models, where the regre...
research
07/12/2018

Orthogonal Matching Pursuit for Text Classification

In text classification, the problem of overfitting arises due to the hig...
research
10/16/2020

Power of FDR Control Methods: The Impact of Ranking Algorithm, Tampered Design, and Symmetric Statistic

As the power of FDR control methods for high-dimensional variable select...
research
07/15/2020

Variable selection in sparse GLARMA models

In this paper, we propose a novel and efficient two-stage variable selec...
research
12/17/2010

Ultra-high Dimensional Multiple Output Learning With Simultaneous Orthogonal Matching Pursuit: A Sure Screening Approach

We propose a novel application of the Simultaneous Orthogonal Matching P...
research
04/29/2012

Optimality of Graphlet Screening in High Dimensional Variable Selection

Consider a linear regression model where the design matrix X has n rows ...
research
03/12/2013

Recovering Non-negative and Combined Sparse Representations

The non-negative solution to an underdetermined linear system can be uni...

Please sign up or login with your details

Forgot password? Click here to reset