Robust Regression and Lasso

Details Robust Regression and Lasso Robust Regression and Lasso

2008-11-11

arxiv.org/abs/0811.1790v1

Computer Science

Information Theory

Scientific paper

Lasso, or $\ell^1$ regularized least squares, has been explored extensively for its remarkable sparsity properties. It is shown in this paper that the solution to Lasso, in addition to its sparsity, has robustness properties: it is the solution to a robust optimization problem. This has two important consequences. First, robustness provides a connection of the regularizer to a physical property, namely, protection from noise. This allows a principled selection of the regularizer, and in particular, generalizations of Lasso that also yield convex optimization problems are obtained by considering different uncertainty sets. Secondly, robustness can itself be used as an avenue to exploring different properties of the solution. In particular, it is shown that robustness of the solution explains why the solution is sparse. The analysis as well as the specific results obtained differ from standard sparsity results, providing different geometric intuition. Furthermore, it is shown that the robust optimization formulation is related to kernel density estimation, and based on this approach, a proof that Lasso is consistent is given using robustness directly. Finally, a theorem saying that sparsity and algorithmic stability contradict each other, and hence Lasso is not stable, is presented.

Affiliated with

Also associated with

No associations

Rating

Robust Regression and Lasso does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Robust Regression and Lasso, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Robust Regression and Lasso will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-100292