UPS Delivers Optimal Phase Diagram in High Dimensional Variable Selection

Details UPS Delivers Optimal Phase Diagram in High Dimensional Variable Selection UPS Delivers Optimal Phase Diagram in High Dimensional Variable Selection

2010-10-25

arxiv.org/abs/1010.5028v1

Mathematics

Statistics Theory

Scientific paper

Consider linear regression in the so-called regime of p much larger than n. We propose the UPS as a new variable selection method. This is a Screen and Clean procedure [Wasserman and Roeder (2009)], in which we screen with the Univariate thresholding, and clean with the Penalized MLE. In many situations, the UPS possesses two important properties: Sure Screening and Separable After Screening (SAS). These properties enable us to reduce the original regression problem to many small-size regression problems that can be fitted separately. We measure the performance of variable selection procedure by the Hamming distance. In many situations, we find that the UPS achieves the optimal rate of convergence, and also yields an optimal partition of the so-called phase diagram. In the two-dimensional phase space calibrated by the signal sparsity and signal strength, there is a three-phase diagram shared by many choices of design matrices. In the first phase, it is possible to recover all signals. In the second phase, exact recovery is impossible, but it is possible to recover most of the signals. In the third phase, successful variable selection is impossible. The UPS is able to recover most of the signals as long as successful variable selection is possible. The lasso and the subset selection are well-known approaches to variable selection. However, somewhat surprisingly, there are regions in the phase space where neither of them is rate optimal, even in very simple cases. The lasso is non-optimal for it is too loose on false positives, and the subset selection is non-optimal for it is too harsh on true positive clusters.

Affiliated with

Also associated with

No associations

Rating

UPS Delivers Optimal Phase Diagram in High Dimensional Variable Selection does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with UPS Delivers Optimal Phase Diagram in High Dimensional Variable Selection, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and UPS Delivers Optimal Phase Diagram in High Dimensional Variable Selection will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-93435