Thresholding-based Iterative Selection Procedures for Model Selection and Shrinkage

Details Thresholding-based Iterative Selection Procedures for Model Selection and Shrinkage Thresholding-based Iterative Selection Procedures for Model Selection and Shrinkage

2008-12-30

arxiv.org/abs/0812.5061v2

Mathematics

Statistics Theory

Submitted to the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics

Scientific paper

This paper discusses a class of thresholding-based iterative selection procedures (TISP) for model selection and shrinkage. People have long before noticed the weakness of the convex $l_1$-constraint (or the soft-thresholding) in wavelets and have designed many different forms of nonconvex penalties to increase model sparsity and accuracy. But for a nonorthogonal regression matrix, there is great difficulty in both investigating the performance in theory and solving the problem in computation. TISP provides a simple and efficient way to tackle this so that we successfully borrow the rich results in the orthogonal design to solve the nonconvex penalized regression for a general design matrix. Our starting point is, however, thresholding rules rather than penalty functions. Indeed, there is a universal connection between them. But a drawback of the latter is its non-unique form, and our approach greatly facilitates the computation and the analysis. In fact, we are able to build the convergence theorem and explore theoretical properties of the selection and estimation via TISP nonasymptotically. More importantly, a novel Hybrid-TISP is proposed based on hard-thresholding and ridge-thresholding. It provides a fusion between the $l_0$-penalty and the $l_2$-penalty, and adaptively achieves the right balance between shrinkage and selection in statistical modeling. In practice, Hybrid-TISP shows superior performance in test-error and is parsimonious.

Affiliated with

Also associated with

No associations

Rating

Thresholding-based Iterative Selection Procedures for Model Selection and Shrinkage does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Thresholding-based Iterative Selection Procedures for Model Selection and Shrinkage, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Thresholding-based Iterative Selection Procedures for Model Selection and Shrinkage will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-117789