Bayesian Optimized Monte Carlo Planning

10/07/2020
by   John Mern, et al.
23

Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. Monte Carlo tree search with progressive widening attempts to improve scaling by sampling from the action space to construct a policy search tree. The performance of progressive widening search is dependent upon the action sampling policy, often requiring problem-specific samplers. In this work, we present a general method for efficient action sampling based on Bayesian optimization. The proposed method uses a Gaussian process to model a belief over the action-value function and selects the action that will maximize the expected improvement in the optimal action value. We implement the proposed approach in a new online tree search algorithm called Bayesian Optimized Monte Carlo Planning (BOMCP). Several experiments show that BOMCP is better able to scale to large action space POMDPs than existing state-of-the-art tree search solvers.

READ FULL TEXT

page 5

page 6

10/07/2020

Improved POMDP Tree Search Planning with Prioritized Action Branching

Online solvers for partially observable Markov decision processes have d...
04/26/2022

An Efficient Dynamic Sampling Policy For Monte Carlo Tree Search

We consider the popular tree-based search strategy within the framework ...
09/23/2021

Adaptive Sampling using POMDPs with Domain-Specific Considerations

We investigate improving Monte Carlo Tree Search based solvers for Parti...
01/25/2020

Bayesian optimization for backpropagation in Monte-Carlo tree search

In large domains, Monte-Carlo tree search (MCTS) is required to estimate...
09/26/2013

Monte-Carlo Planning: Theoretically Fast Convergence Meets Practical Efficiency

Popular Monte-Carlo tree search (MCTS) algorithms for online planning, s...
06/12/2021

Planning Spatial Networks

We tackle the problem of goal-directed graph construction: given a start...
06/17/2022

Efficient Object Manipulation Planning with Monte Carlo Tree Search

This paper presents an efficient approach to object manipulation plannin...

Code Repositories

BOMCP.jl

Bayesian Optimal Monte Carlo Planning POMDP solver


view repo