L1-Penalization for Mixture Regression Models

Details L1-Penalization for Mixture Regression Models L1-Penalization for Mixture Regression Models

2012-02-27

arxiv.org/abs/1202.6046v1

TEST, 2010, Volume 19, 209--285

Statistics

Methodology

This is the author's version of the work (published as a discussion paper in TEST, 2010, Volume 19, 209--285). The final publi

Scientific paper

10.1007/s11749-010-0197-z

We consider a finite mixture of regressions (FMR) model for high-dimensional inhomogeneous data where the number of covariates may be much larger than sample size. We propose an l1-penalized maximum likelihood estimator in an appropriate parameterization. This kind of estimation belongs to a class of problems where optimization and theory for non-convex functions is needed. This distinguishes itself very clearly from high-dimensional estimation with convex loss- or objective functions, as for example with the Lasso in linear or generalized linear models. Mixture models represent a prime and important example where non-convexity arises. For FMR models, we develop an efficient EM algorithm for numerical optimization with provable convergence properties. Our penalized estimator is numerically better posed (e.g., boundedness of the criterion function) than unpenalized maximum likelihood estimation, and it allows for effective statistical regularization including variable selection. We also present some asymptotic theory and oracle inequalities: due to non-convexity of the negative log-likelihood function, different mathematical arguments are needed than for problems with convex losses. Finally, we apply the new method to both simulated and real data.

Affiliated with

Also associated with

No associations

Rating

L1-Penalization for Mixture Regression Models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with L1-Penalization for Mixture Regression Models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and L1-Penalization for Mixture Regression Models will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-605039