Statistics – Computation
Scientific paper
2012-01-17
Statistics
Computation
28 pages, 5 figures
Scientific paper
Regularization is widely used in statistics and machine learning to prevent overfitting and gear solution towards prior information. In general, a regularized estimation problem minimizes the sum of a loss function and a penalty term. The penalty term is usually weighted by a tuning parameter and encourages certain constraints on the parameters to be estimated. Particular choices of constraints lead to the popular lasso, fused-lasso, and other generalized $l_1$ penalized regression methods. Although there has been a lot of research in this area, developing efficient optimization methods for many nonseparable penalties remains a challenge. In this article we propose an exact path solver based on ordinary differential equations (EPSODE) that works for any convex loss function and can deal with generalized $l_1$ penalties as well as more complicated regularization such as inequality constraints encountered in shape-restricted regressions and nonparametric density estimation. In the path following process, the solution path hits, exits, and slides along the various constraints and vividly illustrates the tradeoffs between goodness of fit and model parsimony. In practice, the EPSODE can be coupled with AIC, BIC, $C_p$ or cross-validation to select an optimal tuning parameter. Our applications to generalized $l_1$ regularized generalized linear models, shape-restricted regressions, Gaussian graphical models, and nonparametric density estimation showcase the potential of the EPSODE algorithm.
Wu Yichao
Zhou Hua
No associations
LandOfFree
A Generic Path Algorithm for Regularized Statistical Estimation does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with A Generic Path Algorithm for Regularized Statistical Estimation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Generic Path Algorithm for Regularized Statistical Estimation will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-97951