Orthogonalizing Penalized Regression

Details Orthogonalizing Penalized Regression Orthogonalizing Penalized Regression

2011-07-31

arxiv.org/abs/1108.0185v1

Statistics

Computation

Scientific paper

Since the penalized likelihood function of the smoothly clipped absolute deviation (SCAD) penalty is highly non-linear and has many local optima, finding a local solution to achieve the so-called oracle property is an open problem. We propose an iterative algorithm, called the OEM algorithm, to fill this gap. The development of the algorithm draws direct impetus from a missing-data problem arising in design of experiments with an orthogonal complete matrix. In each iteration, the algorithm imputes the missing data based on the current estimates of the parameters and updates a closed-form solution associated with the complete data. By introducing a procedure called active orthogonization, we make the algorithm broadly applicable to problems with arbitrary regression matrices. In addition to the SCAD penalty, the proposed algorithm works for other penalties like the MCP, lasso and nonnegative garrote. Convergence and convergence rate of the algorithm are examined. The algorithm has several unique theoretical properties. For the SCAD and MCP penalties, an OEM sequence can achieve the oracle property after sufficient iterations. For various penalties, an OEM sequence converges to a point having grouping coherence for fully aliased regression matrices. For computing the ordinary least squares estimator with a singular regression matrix, an OEM sequence converges to the Moore-Penrose generalized inverse-based least squares estimator.

Affiliated with

Also associated with

No associations

Rating

Orthogonalizing Penalized Regression does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Orthogonalizing Penalized Regression, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Orthogonalizing Penalized Regression will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-703309