Bayesian Optimized Monte Carlo Planning

by   John Mern, et al.

Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. Monte Carlo tree search with progressive widening attempts to improve scaling by sampling from the action space to construct a policy search tree. The performance of progressive widening search is dependent upon the action sampling policy, often requiring problem-specific samplers. In this work, we present a general method for efficient action sampling based on Bayesian optimization. The proposed method uses a Gaussian process to model a belief over the action-value function and selects the action that will maximize the expected improvement in the optimal action value. We implement the proposed approach in a new online tree search algorithm called Bayesian Optimized Monte Carlo Planning (BOMCP). Several experiments show that BOMCP is better able to scale to large action space POMDPs than existing state-of-the-art tree search solvers.


page 5

page 6


Improved POMDP Tree Search Planning with Prioritized Action Branching

Online solvers for partially observable Markov decision processes have d...

An Efficient Dynamic Sampling Policy For Monte Carlo Tree Search

We consider the popular tree-based search strategy within the framework ...

Adaptive Sampling using POMDPs with Domain-Specific Considerations

We investigate improving Monte Carlo Tree Search based solvers for Parti...

Bayesian optimization for backpropagation in Monte-Carlo tree search

In large domains, Monte-Carlo tree search (MCTS) is required to estimate...

Monte-Carlo Planning: Theoretically Fast Convergence Meets Practical Efficiency

Popular Monte-Carlo tree search (MCTS) algorithms for online planning, s...

Planning Spatial Networks

We tackle the problem of goal-directed graph construction: given a start...

Efficient Object Manipulation Planning with Monte Carlo Tree Search

This paper presents an efficient approach to object manipulation plannin...

Code Repositories


Bayesian Optimal Monte Carlo Planning POMDP solver

view repo