Monte-Carlo Planning: Theoretically Fast Convergence Meets Practical Efficiency

by   Zohar Feldman, et al.

Popular Monte-Carlo tree search (MCTS) algorithms for online planning, such as epsilon-greedy tree search and UCT, aim at rapidly identifying a reasonably good action, but provide rather poor worst-case guarantees on performance improvement over time. In contrast, a recently introduced MCTS algorithm BRUE guarantees exponential-rate improvement over time, yet it is not geared towards identifying reasonably good choices right at the go. We take a stand on the individual strengths of these two classes of algorithms, and show how they can be effectively connected. We then rationalize a principle of "selective tree expansion", and suggest a concrete implementation of this principle within MCTS. The resulting algorithm,s favorably compete with other MCTS algorithms under short planning times, while preserving the attractive convergence properties of BRUE.


page 4

page 9


Bayesian Optimized Monte Carlo Planning

Online solvers for partially observable Markov decision processes have d...

Monte Carlo Tree Search Based Tactical Maneuvering

In this paper we explore the application of simultaneous move Monte Carl...

Cooperative Driving at Unsignalized Intersections Using Tree Search

In this paper, we propose a new cooperative driving strategy for connect...

Convex Regularization in Monte-Carlo Tree Search

Monte-Carlo planning and Reinforcement Learning (RL) are essential to se...

Simple Regret Optimization in Online Planning for Markov Decision Processes

We consider online planning in Markov decision processes (MDPs). In onli...

Efficient Object Manipulation Planning with Monte Carlo Tree Search

This paper presents an efficient approach to object manipulation plannin...

TreeGen – a Monte Carlo generator for data frames

The typical problem in Data Science is creating a structure that encodes...