Bayesian Inference in Monte-Carlo Tree Search

03/15/2012
by   Gerald Tesauro, et al.
0

Monte-Carlo Tree Search (MCTS) methods are drawing great interest after yielding breakthrough results in computer Go. This paper proposes a Bayesian approach to MCTS that is inspired by distributionfree approaches such as UCT [13], yet significantly differs in important respects. The Bayesian framework allows potentially much more accurate (Bayes-optimal) estimation of node values and node uncertainties from a limited number of simulation trials. We further propose propagating inference in the tree via fast analytic Gaussian approximation methods: this can make the overhead of Bayesian inference manageable in domains such as Go, while preserving high accuracy of expected-value estimates. We find substantial empirical outperformance of UCT in an idealized bandit-tree test environment, where we can obtain valuable insights by comparing with known ground truth. Additionally we rigorously prove on-policy and off-policy convergence of the proposed methods.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/14/2018

Learning in POMDPs with Monte Carlo Tree Search

The POMDP is a powerful framework for reasoning under outcome and inform...
10/07/2020

Bayesian Optimized Monte Carlo Planning

Online solvers for partially observable Markov decision processes have d...
01/25/2020

Bayesian optimization for backpropagation in Monte-Carlo tree search

In large domains, Monte-Carlo tree search (MCTS) is required to estimate...
03/27/2013

Heuristic Search as Evidential Reasoning

BPS, the Bayesian Problem Solver, applies probabilistic inference and de...
12/08/2020

Bridging Bayesian, frequentist and fiducial (BFF) inferences using confidence distribution

Bayesian, frequentist and fiducial (BFF) inferences are much more congru...
06/15/2020

Variational Bayesian Monte Carlo with Noisy Likelihoods

Variational Bayesian Monte Carlo (VBMC) is a recently introduced framewo...
02/14/2021

Costly Features Classification using Monte Carlo Tree Search

We consider the problem of costly feature classification, where we seque...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.