Bayesian Inference in Monte-Carlo Tree Search

Details Bayesian Inference in Monte-Carlo Tree Search Bayesian Inference in Monte-Carlo Tree Search

2012-03-15

arxiv.org/abs/1203.3519v1

Computer Science

Learning

Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI2010)

Scientific paper

Monte-Carlo Tree Search (MCTS) methods are drawing great interest after yielding breakthrough results in computer Go. This paper proposes a Bayesian approach to MCTS that is inspired by distributionfree approaches such as UCT [13], yet significantly differs in important respects. The Bayesian framework allows potentially much more accurate (Bayes-optimal) estimation of node values and node uncertainties from a limited number of simulation trials. We further propose propagating inference in the tree via fast analytic Gaussian approximation methods: this can make the overhead of Bayesian inference manageable in domains such as Go, while preserving high accuracy of expected-value estimates. We find substantial empirical outperformance of UCT in an idealized bandit-tree test environment, where we can obtain valuable insights by comparing with known ground truth. Additionally we rigorously prove on-policy and off-policy convergence of the proposed methods.

Affiliated with

Also associated with

No associations

Rating

Bayesian Inference in Monte-Carlo Tree Search does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Bayesian Inference in Monte-Carlo Tree Search, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Bayesian Inference in Monte-Carlo Tree Search will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-32360