Safe Feature Elimination in Sparse Supervised Learning

Details Safe Feature Elimination in Sparse Supervised Learning Safe Feature Elimination in Sparse Supervised Learning

2010-09-17

arxiv.org/abs/1009.3515v2

Computer Science

Learning

New version is on arXiv:1009.4219

Scientific paper

We investigate fast methods that allow to quickly eliminate variables (features) in supervised learning problems involving a convex loss function and a $l_1$-norm penalty, leading to a potentially substantial reduction in the number of variables prior to running the supervised learning algorithm. The methods are not heuristic: they only eliminate features that are {\em guaranteed} to be absent after solving the learning problem. Our framework applies to a large class of problems, including support vector machine classification, logistic regression and least-squares. The complexity of the feature elimination step is negligible compared to the typical computational effort involved in the sparse supervised learning problem: it grows linearly with the number of features times the number of examples, with much better count if data is sparse. We apply our method to data sets arising in text classification and observe a dramatic reduction of the dimensionality, hence in computational effort required to solve the learning problem, especially when very sparse classifiers are sought. Our method allows to immediately extend the scope of existing algorithms, allowing us to run them on data sets of sizes that were out of their reach before.

Affiliated with

Also associated with

No associations

Rating

Safe Feature Elimination in Sparse Supervised Learning does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Safe Feature Elimination in Sparse Supervised Learning, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Safe Feature Elimination in Sparse Supervised Learning will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-476500