A Majorization-Minimization Approach to Variable Selection Using Spike and Slab Priors

Details A Majorization-Minimization Approach to Variable Selection Using Spike and Slab Priors A Majorization-Minimization Approach to Variable Selection Using Spike and Slab Priors

2010-05-06

arxiv.org/abs/1005.0891v1

Statistics

Methodology

67 pages, 8 figures, 6 tables.

Scientific paper

We develop a method to carry out MAP estimation for a class of Bayesian regression models in which coefficients are assigned with Gaussian-based spike and slab priors weighted by Bernoulli variables. Unlike simulation-based inference methods, the proposed method directly optimizes the logarithm of the joint posterior density for parameter estimation. The corresponding optimization problem has an objective function in Lagrangian form in that regression coefficients are regularized by a mixture of squared $l_{2}$ and $l_{0}$ norms. A tight approximation to the $l_{0}$ norm using majorization-minimization techniques is derived, and a coordinate descent algorithm in conjunction with a specified soft-thresholding scheme is used in searching for the optimizer of the approximated objective. Simulation studies show that the proposed method can lead to more accurate variable selection than other benchmark methods. It also shows that the Irrepresentable Condition (Zhao and Yu, 2006) appears to have less impacts on the performance of the proposed method. Theoretical results further show that under some regular conditions, sign consistency can always be established, even when the Irrepresentable Condition is violated. Results on posterior model consistency and estimation consistency, and an extension to parameter estimation in the Generalized Linear Models are provided.

Affiliated with

Also associated with

No associations

Rating

A Majorization-Minimization Approach to Variable Selection Using Spike and Slab Priors does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A Majorization-Minimization Approach to Variable Selection Using Spike and Slab Priors, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Majorization-Minimization Approach to Variable Selection Using Spike and Slab Priors will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-25641