Dense Signals, Linear Estimators, and Out-of-Sample Prediction for High-Dimensional Linear Models

Details Dense Signals, Linear Estimators, and Out-of-Sample Prediction for High-Dimensional Linear Models Dense Signals, Linear Estimators, and Out-of-Sample Prediction for High-Dimensional Linear Models

2011-02-15

arxiv.org/abs/1102.2952v3

Mathematics

Statistics Theory

43 pages, 3 figures

Scientific paper

Motivated by questions about dense (non-sparse) signals in high-dimensional data analysis, we study the unconditional out-of-sample prediction error (predictive risk) associated with three popular linear estimators for high-dimensional linear models: ridge regression estimators, scalar multiples of the ordinary least squares (OLS) estimator (referred to as James-Stein shrinkage estimators), and marginal regression estimators. The results in this paper require no assumptions about sparsity and imply: (i) if prior information about the population predictor covariance is available, then the ridge estimator outperforms the OLS, James-Stein, and marginal estimators; (ii) if little is known about the population predictor covariance, then the James-Stein estimator may be an effective alternative to the ridge estimator; and (iii) the marginal estimator has serious deficiencies for out-of-sample prediction. Both finite sample and asymptotic properties of the estimators are studied in this paper. Though various asymptotic regimes are considered, we focus on the setting where the number of predictors is roughly proportional to the number of observations. Ultimately, the results presented here provide new and detailed practical guidance regarding several well-known non-sparse methods for high-dimensional linear models.

Affiliated with

Also associated with

No associations

Rating

Dense Signals, Linear Estimators, and Out-of-Sample Prediction for High-Dimensional Linear Models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Dense Signals, Linear Estimators, and Out-of-Sample Prediction for High-Dimensional Linear Models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dense Signals, Linear Estimators, and Out-of-Sample Prediction for High-Dimensional Linear Models will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-377794