Learning unbelievable marginal probabilities

Details Learning unbelievable marginal probabilities Learning unbelievable marginal probabilities

2011-06-02

arxiv.org/abs/1106.0483v1

Computer Science

Artificial Intelligence

10 pages, 3 figures, submitted to NIPS*2011

Scientific paper

Loopy belief propagation performs approximate inference on graphical models with loops. One might hope to compensate for the approximation by adjusting model parameters. Learning algorithms for this purpose have been explored previously, and the claim has been made that every set of locally consistent marginals can arise from belief propagation run on a graphical model. On the contrary, here we show that many probability distributions have marginals that cannot be reached by belief propagation using any set of model parameters or any learning algorithm. We call such marginals `unbelievable.' This problem occurs whenever the Hessian of the Bethe free energy is not positive-definite at the target marginals. All learning algorithms for belief propagation necessarily fail in these cases, producing beliefs or sets of beliefs that may even be worse than the pre-learning approximation. We then show that averaging inaccurate beliefs, each obtained from belief propagation using model parameters perturbed about some learned mean values, can achieve the unbelievable marginals.

Affiliated with

Also associated with

No associations

Rating

Learning unbelievable marginal probabilities does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Learning unbelievable marginal probabilities, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Learning unbelievable marginal probabilities will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-495608