The sparsity and bias of the Lasso selection in high-dimensional linear regression

Details The sparsity and bias of the Lasso selection in high-dimensional linear regression The sparsity and bias of the Lasso selection in high-dimensional linear regression

2008-08-07

arxiv.org/abs/0808.0967v1

Annals of Statistics 2008, Vol. 36, No. 4, 1567-1594

Mathematics

Statistics Theory

Published in at http://dx.doi.org/10.1214/07-AOS520 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of

Scientific paper

10.1214/07-AOS520

Meinshausen and Buhlmann [Ann. Statist. 34 (2006) 1436--1462] showed that, for neighborhood selection in Gaussian graphical models, under a neighborhood stability condition, the LASSO is consistent, even when the number of variables is of greater order than the sample size. Zhao and Yu [(2006) J. Machine Learning Research 7 2541--2567] formalized the neighborhood stability condition in the context of linear regression as a strong irrepresentable condition. That paper showed that under this condition, the LASSO selects exactly the set of nonzero regression coefficients, provided that these coefficients are bounded away from zero at a certain rate. In this paper, the regression coefficients outside an ideal model are assumed to be small, but not necessarily zero. Under a sparse Riesz condition on the correlation of design variables, we prove that the LASSO selects a model of the correct order of dimensionality, controls the bias of the selected model at a level determined by the contributions of small regression coefficients and threshold bias, and selects all coefficients of greater order than the bias of the selected model. Moreover, as a consequence of this rate consistency of the LASSO in model selection, it is proved that the sum of error squares for the mean response and the $\ell_{\alpha}$-loss for the regression coefficients converge at the best possible rates under the given conditions. An interesting aspect of our results is that the logarithm of the number of variables can be of the same order as the sample size for certain random dependent designs.

Affiliated with

Also associated with

No associations

Rating

The sparsity and bias of the Lasso selection in high-dimensional linear regression does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with The sparsity and bias of the Lasso selection in high-dimensional linear regression, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The sparsity and bias of the Lasso selection in high-dimensional linear regression will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-213275