DeepAI AI Chat
Log In Sign Up

Lookahead Pathology in Monte-Carlo Tree Search

12/10/2022
by   Khoi P. N. Nguyen, et al.
0

Monte-Carlo Tree Search (MCTS) is an adversarial search paradigm that first found prominence with its success in the domain of computer Go. Early theoretical work established the game-theoretic soundness and convergence bounds for Upper Confidence bounds applied to Trees (UCT), the most popular instantiation of MCTS; however, there remain notable gaps in our understanding of how UCT behaves in practice. In this work, we address one such gap by considering the question of whether UCT can exhibit lookahead pathology – a paradoxical phenomenon first observed in Minimax search where greater search effort leads to worse decision-making. We introduce a novel family of synthetic games that offer rich modeling possibilities while remaining amenable to mathematical analysis. Our theoretical and experimental results suggest that UCT is indeed susceptible to pathological behavior in a range of games drawn from this family.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/08/2022

Combining Monte-Carlo Tree Search with Proof-Number Search

Proof-Number Search (PNS) and Monte-Carlo Tree Search (MCTS) have been s...
03/16/2023

Proof Number Based Monte-Carlo Tree Search

This paper proposes a new game search algorithm, PN-MCTS, that combines ...
05/11/2015

Adapting Improved Upper Confidence Bounds for Monte-Carlo Tree Search

The UCT algorithm, which combines the UCB algorithm and Monte-Carlo Tree...
03/15/2012

Understanding Sampling Style Adversarial Search Methods

UCT has recently emerged as an exciting new adversarial reasoning techni...
02/16/2019

Re-determinizing Information Set Monte Carlo Tree Search in Hanabi

This technical report documents the winner of the Computational Intellig...
06/15/2020

On Effective Parallelization of Monte Carlo Tree Search

Despite its groundbreaking success in Go and computer games, Monte Carlo...