Coordinate descent algorithms for lasso penalized regression

Details Coordinate descent algorithms for lasso penalized regression Coordinate descent algorithms for lasso penalized regression

2008-03-27

arxiv.org/abs/0803.3876v1

Annals of Applied Statistics 2008, Vol. 2, No. 1, 224-244

Statistics

Applications

Published in at http://dx.doi.org/10.1214/07-AOAS147 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Ins

Scientific paper

10.1214/07-AOAS147

Imposition of a lasso penalty shrinks parameter estimates toward zero and performs continuous model selection. Lasso penalized regression is capable of handling linear regression problems where the number of predictors far exceeds the number of cases. This paper tests two exceptionally fast algorithms for estimating regression coefficients with a lasso penalty. The previously known $\ell_2$ algorithm is based on cyclic coordinate descent. Our new $\ell_1$ algorithm is based on greedy coordinate descent and Edgeworth's algorithm for ordinary $\ell_1$ regression. Each algorithm relies on a tuning constant that can be chosen by cross-validation. In some regression problems it is natural to group parameters and penalize parameters group by group rather than separately. If the group penalty is proportional to the Euclidean norm of the parameters of the group, then it is possible to majorize the norm and reduce parameter estimation to $\ell_2$ regression with a lasso penalty. Thus, the existing algorithm can be extended to novel settings. Each of the algorithms discussed is tested via either simulated or real data or both. The Appendix proves that a greedy form of the $\ell_2$ algorithm converges to the minimum value of the objective function.

Affiliated with

Also associated with

No associations

Rating

Coordinate descent algorithms for lasso penalized regression does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Coordinate descent algorithms for lasso penalized regression, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Coordinate descent algorithms for lasso penalized regression will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-353741