DeepAI AI Chat
Log In Sign Up

Lookahead Pathology in Monte-Carlo Tree Search

by   Khoi P. N. Nguyen, et al.

Monte-Carlo Tree Search (MCTS) is an adversarial search paradigm that first found prominence with its success in the domain of computer Go. Early theoretical work established the game-theoretic soundness and convergence bounds for Upper Confidence bounds applied to Trees (UCT), the most popular instantiation of MCTS; however, there remain notable gaps in our understanding of how UCT behaves in practice. In this work, we address one such gap by considering the question of whether UCT can exhibit lookahead pathology – a paradoxical phenomenon first observed in Minimax search where greater search effort leads to worse decision-making. We introduce a novel family of synthetic games that offer rich modeling possibilities while remaining amenable to mathematical analysis. Our theoretical and experimental results suggest that UCT is indeed susceptible to pathological behavior in a range of games drawn from this family.


page 1

page 2

page 3

page 4


Combining Monte-Carlo Tree Search with Proof-Number Search

Proof-Number Search (PNS) and Monte-Carlo Tree Search (MCTS) have been s...

Proof Number Based Monte-Carlo Tree Search

This paper proposes a new game search algorithm, PN-MCTS, that combines ...

Adapting Improved Upper Confidence Bounds for Monte-Carlo Tree Search

The UCT algorithm, which combines the UCB algorithm and Monte-Carlo Tree...

Understanding Sampling Style Adversarial Search Methods

UCT has recently emerged as an exciting new adversarial reasoning techni...

Re-determinizing Information Set Monte Carlo Tree Search in Hanabi

This technical report documents the winner of the Computational Intellig...

On Effective Parallelization of Monte Carlo Tree Search

Despite its groundbreaking success in Go and computer games, Monte Carlo...