Monte Carlo Tree Search with Heuristic Evaluations using Implicit Minimax Backups

by   Marc Lanctot, et al.
Maastricht University
University of Denver

Monte Carlo Tree Search (MCTS) has improved the performance of game engines in domains such as Go, Hex, and general game playing. MCTS has been shown to outperform classic alpha-beta search in games where good heuristic evaluations are difficult to obtain. In recent years, combining ideas from traditional minimax search in MCTS has been shown to be advantageous in some domains, such as Lines of Action, Amazons, and Breakthrough. In this paper, we propose a new way to use heuristic evaluations to guide the MCTS search by storing the two sources of information, estimated win rates and heuristic evaluations, separately. Rather than using the heuristic evaluations to replace the playouts, our technique backs them up implicitly during the MCTS simulations. These minimax values are then used to guide future simulations. We show that using implicit minimax backups leads to stronger play performance in Kalah, Breakthrough, and Lines of Action.


Proof Number Based Monte-Carlo Tree Search

This paper proposes a new game search algorithm, PN-MCTS, that combines ...

Understanding Sampling Style Adversarial Search Methods

UCT has recently emerged as an exciting new adversarial reasoning techni...

Predicting The Performance of Minimax and Product in Game-Tree

The discovery that the minimax decision rule performs poorly in some gam...

A New Challenge: Approaching Tetris Link with AI

Decades of research have been invested in making computer programs for p...

Information and Search in Computer Chess

The article describes a model of chess based on information theory. A ma...

Neural Networks for Chess

AlphaZero, Leela Chess Zero and Stockfish NNUE revolutionized Computer C...

A Cure for Pathological Behavior in Games that Use Minimax

The traditional approach to choosing moves in game-playing programs is t...

Please sign up or login with your details

Forgot password? Click here to reset