DeepAI AI Chat
Log In Sign Up

On Effective Parallelization of Monte Carlo Tree Search

by   Anji Liu, et al.

Despite its groundbreaking success in Go and computer games, Monte Carlo Tree Search (MCTS) is computationally expensive as it requires a substantial number of rollouts to construct the search tree, which calls for effective parallelization. However, how to design effective parallel MCTS algorithms has not been systematically studied and remains poorly understood. In this paper, we seek to lay its first theoretical foundations, by examining the potential performance loss caused by parallelization when achieving a desired speedup. In particular, we focus on studying the conditions under which the performance loss (measured in excess regret) vanishes over time. To this end, we propose a general parallel MCTS framework that can be specialized to major existing parallel MCTS algorithms. We derive two necessary conditions for the algorithms covered by the general framework to have vanishing excess regret (i.e. excess regret converges to zero as the total number of rollouts grows). We demonstrate the effectiveness of the necessary conditions by showing that, for depth-2 search trees, the recently developed WU-UCT algorithm satisfies both necessary conditions and has provable vanishing excess regret. Finally, we perform empirical studies to closely examine the necessary conditions under the general tree search setting (with arbitrary tree depth). It shows that the topological discrepancy between the search trees constructed by the parallel and the sequential MCTS algorithms is the main reason for the performance loss.


page 1

page 2

page 3

page 4


P-MCGS: Parallel Monte Carlo Acyclic Graph Search

Recently, there have been great interests in Monte Carlo Tree Search (MC...

Parallelization of Monte Carlo Tree Search in Continuous Domains

Monte Carlo Tree Search (MCTS) has proven to be capable of solving chall...

Adapting Improved Upper Confidence Bounds for Monte-Carlo Tree Search

The UCT algorithm, which combines the UCB algorithm and Monte-Carlo Tree...

Structured Parallel Programming for Monte Carlo Tree Search

In this paper, we present a new algorithm for parallel Monte Carlo tree ...

On the convergence of dynamic implementations of Hamiltonian Monte Carlo and No U-Turn Samplers

There is substantial empirical evidence about the success of dynamic imp...

Monte-Carlo Tree Search by Best Arm Identification

Recent advances in bandit tools and techniques for sequential learning a...