Statistics – Methodology
Scientific paper
2011-09-11
Statistics
Methodology
Scientific paper
We develop the Bayesian bridge estimator for regularized regression and classification. We focus on two key mixture representations for the prior distribution that give rise to the Bayesian bridge model: (1) a scale mixture of normals with respect to an alpha-stable random variable; and (2) a mixture of Bartlett--Fejer kernels (or triangle densities) with respect to a two-component mixture of gamma random variables. The first representation is a well known result due to West (1987). We show how it can be exploited to yield an efficient, parallelizable EM algorithm for calculating the joint posterior mode---even under certain non-Gaussian likelihoods, such as those that arise in logistic or quantile regression. The second representation is new. Its main virtue is that avoids the need to deal with exponentially tilted stable random variables, and therefore leads to a much simpler MCMC scheme than the scale mixture representation. It also provides insight into the multimodality of the joint posterior distribution, which is a notable feature of the Bayesian bridge model that is absent under more-traditional ridge or lasso-type priors. Finally, we show how our approach can be extended to a wider class of non-convex regularization penalties based on Polya-type characteristic functions, including the double-Pareto model of Armagan et. al. (2011). We compare the performance of the Bayesian bridge model to its classical cousin across a variety of data sets, both simulated and real.
Polson Nicholas G.
Scott James G.
No associations
LandOfFree
The Bayesian Bridge does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with The Bayesian Bridge, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The Bayesian Bridge will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-19852