Monte-Carlo Planning: Theoretically Fast Convergence Meets Practical Efficiency

09/26/2013
by   Zohar Feldman, et al.
0

Popular Monte-Carlo tree search (MCTS) algorithms for online planning, such as epsilon-greedy tree search and UCT, aim at rapidly identifying a reasonably good action, but provide rather poor worst-case guarantees on performance improvement over time. In contrast, a recently introduced MCTS algorithm BRUE guarantees exponential-rate improvement over time, yet it is not geared towards identifying reasonably good choices right at the go. We take a stand on the individual strengths of these two classes of algorithms, and show how they can be effectively connected. We then rationalize a principle of "selective tree expansion", and suggest a concrete implementation of this principle within MCTS. The resulting algorithm,s favorably compete with other MCTS algorithms under short planning times, while preserving the attractive convergence properties of BRUE.

READ FULL TEXT

page 4

page 9

10/07/2020

Bayesian Optimized Monte Carlo Planning

Online solvers for partially observable Markov decision processes have d...
09/13/2020

Monte Carlo Tree Search Based Tactical Maneuvering

In this paper we explore the application of simultaneous move Monte Carl...
02/04/2019

Cooperative Driving at Unsignalized Intersections Using Tree Search

In this paper, we propose a new cooperative driving strategy for connect...
07/01/2020

Convex Regularization in Monte-Carlo Tree Search

Monte-Carlo planning and Reinforcement Learning (RL) are essential to se...
06/15/2012

Simple Regret Optimization in Online Planning for Markov Decision Processes

We consider online planning in Markov decision processes (MDPs). In onli...
06/17/2022

Efficient Object Manipulation Planning with Monte Carlo Tree Search

This paper presents an efficient approach to object manipulation plannin...
11/17/2020

TreeGen – a Monte Carlo generator for data frames

The typical problem in Data Science is creating a structure that encodes...