EigenNet: A Bayesian hybrid of generative and conditional models for sparse learning

Details EigenNet: A Bayesian hybrid of generative and conditional models for sparse learning EigenNet: A Bayesian hybrid of generative and conditional models for sparse learning

2011-02-04

arxiv.org/abs/1102.0836v2

Computer Science

Learning

Scientific paper

It is a challenging task to select correlated variables in a high dimensional space. To address this challenge, the elastic net has been developed and successfully applied to many applications. Despite its great success, the elastic net does not explicitly use correlation information embedded in data to select correlated variables. To overcome this limitation, we present a novel Bayesian hybrid model, the EigenNet, that uses the eigenstructures of data to guide variable selection. Specifically, it integrates a sparse conditional classification model with a generative model capturing variable correlations in a principled Bayesian framework. We reparameterize the hybrid model in the eigenspace to avoid overfiting and to increase the computational efficiency of its MCMC sampler. Furthermore, we provide an alternative view to the EigenNet from a regularization perspective: the EigenNet has an adaptive eigenspace-based composite regularizer, which naturally generalizes the $l_{1/2}$ regularizer used by the elastic net. Experiments on synthetic and real data show that the EigenNet significantly outperforms the lasso, the elastic net, and the Bayesian lasso in terms of prediction accuracy, especially when the number of training samples is smaller than the number of variables.

Affiliated with

Also associated with

No associations

Rating

EigenNet: A Bayesian hybrid of generative and conditional models for sparse learning does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with EigenNet: A Bayesian hybrid of generative and conditional models for sparse learning, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and EigenNet: A Bayesian hybrid of generative and conditional models for sparse learning will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-491536