Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression

Details Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression

2008-10-09

arxiv.org/abs/0810.1644v1

Mathematics

Statistics Theory

Submitted to the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics

Scientific paper

We study the problem of high-dimensional variable selection via some two-step procedures. First we show that given some good initial estimator which is $\ell_{\infty}$-consistent but not necessarily variable selection consistent, we can apply the nonnegative Garrote, adaptive Lasso or hard-thresholding procedure to obtain a final estimator that is both estimation and variable selection consistent. Unlike the Lasso, our results do not require the irrepresentable condition which could fail easily even for moderate $p_n$ (Zhao and Yu, 2007) and it also allows $p_n$ to grow almost as fast as $\exp(n)$ (for hard-thresholding there is no restriction on $p_n$). We also study the conditions under which the Ridge regression can be used as an initial estimator. We show that under a relaxed identifiable condition, the Ridge estimator is $\ell_{\infty}$-consistent. Such a condition is usually satisfied when $p_n\le n$ and does not require the partial orthogonality between relevant and irrelevant covariates which is needed for the univariate regression in (Huang et al., 2008). Our numerical studies show that when using the Lasso or Ridge as initial estimator, the two-step procedures have a higher sparsity recovery rate than the Lasso or adaptive Lasso with univariate regression used in (Huang et al., 2008).

Affiliated with

Also associated with

No associations

Rating

Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Some Two-Step Procedures for Variable Selection in High-Dimensional Linear Regression will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-460908