Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction

Details Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction

2011-12-16

arxiv.org/abs/1112.3714v1

Computer Science

Learning

Preprint submitted to Machine Learning Journal

Scientific paper

We show how to incorporate information from labeled examples into nonnegative matrix factorization (NMF), a popular unsupervised learning algorithm for dimensionality reduction. In addition to mapping the data into a space of lower dimensionality, our approach aims to preserve the nonnegative components of the data that are important for classification. We identify these components from the support vectors of large-margin classifiers and derive iterative updates to preserve them in a semi-supervised version of NMF. These updates have a simple multiplicative form like their unsupervised counterparts; they are also guaranteed at each iteration to decrease their loss function---a weighted sum of I-divergences that captures the trade-off between unsupervised and supervised learning. We evaluate these updates for dimensionality reduction when they are used as a precursor to linear classification. In this role, we find that they yield much better performance than their unsupervised counterparts. We also find one unexpected benefit of the low dimensional representations discovered by our approach: often they yield more accurate classifiers than both ordinary and transductive SVMs trained in the original input space.

Affiliated with

Also associated with

No associations

Rating

Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-136396