The Variational Garrote

Details The Variational Garrote The Variational Garrote

2011-09-02

arxiv.org/abs/1109.0486v2

Statistics

Methodology

17 pages, 8 figures

Scientific paper

In this paper, I present a new solution method for sparse regression using L0 regularization. The model introduces a sparseness mechanism in the likelihood, instead of in the prior, as is done in the spike and slab model. The posterior probability is computed in the variational approximation. The variational parameters appear in the approximate model in a way that is similar to Breiman's Garrote model. I refer to this method as the variational Garrote (VG). The VG is compared numerically with the Lasso method and with ridge regression. Numerical results on synthetic data show that the VG yields more accurate predictions and more accurately reconstructs the true model than the other methods. The naive implementation of the VG scales cubic with the number of features. By introducing Lagrange multipliers we obtain a dual formulation of the problem that scales cubic in the number of samples, but close to linear in the number of features.

Affiliated with

Also associated with

No associations

Rating

The Variational Garrote does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with The Variational Garrote, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The Variational Garrote will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-688204