Mathematics – Statistics Theory
Scientific paper
2007-03-28
Mathematics
Statistics Theory
Scientific paper
We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function, yielding a ``pre-conditioned'' response variable. The primary method used for this initial regression is supervised principal components. Then we apply a standard procedure such as forward stepwise selection or the LASSO to the pre-conditioned response variable. In a number of simulated and real data examples, this two-step procedure outperforms forward stepwise selection or the usual LASSO (applied directly to the raw outcome). We also show that under a certain Gaussian latent variable model, application of the LASSO to the pre-conditioned response variable is consistent as the number of predictors and observations increases. Moreover, when the observational noise is rather large, the suggested procedure can give a more accurate estimate than LASSO. We illustrate our method on some real problems, including survival analysis with microarray data.
Bair Eric
Hastie Trevor
Paul Debashis
Tibshirani Robert
No associations
LandOfFree
"Pre-conditioning" for feature selection and regression in high-dimensional problems does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with "Pre-conditioning" for feature selection and regression in high-dimensional problems, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and "Pre-conditioning" for feature selection and regression in high-dimensional problems will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-711228