Monte-Carlo Graph Search for AlphaZero

12/20/2020
by   Johannes Czech, et al.
121

The AlphaZero algorithm has been successfully applied in a range of discrete domains, most notably board games. It utilizes a neural network, that learns a value and policy function to guide the exploration in a Monte-Carlo Tree Search. Although many search improvements have been proposed for Monte-Carlo Tree Search in the past, most of them refer to an older variant of the Upper Confidence bounds for Trees algorithm that does not use a policy for planning. We introduce a new, improved search algorithm for AlphaZero which generalizes the search tree to a directed acyclic graph. This enables information flow across different subtrees and greatly reduces memory consumption. Along with Monte-Carlo Graph Search, we propose a number of further extensions, such as the inclusion of Epsilon-greedy exploration, a revised terminal solver and the integration of domain knowledge as constraints. In our evaluations, we use the CrazyAra engine on chess and crazyhouse as examples to show that these changes bring significant improvements to AlphaZero.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/08/2022

Combining Monte-Carlo Tree Search with Proof-Number Search

Proof-Number Search (PNS) and Monte-Carlo Tree Search (MCTS) have been s...
05/11/2015

Adapting Improved Upper Confidence Bounds for Monte-Carlo Tree Search

The UCT algorithm, which combines the UCB algorithm and Monte-Carlo Tree...
03/21/2021

Dual Monte Carlo Tree Search

AlphaZero, using a combination of Deep Neural Networks and Monte Carlo T...
11/19/2018

Feature selection as Monte-Carlo Search in Growing Single Rooted Directed Acyclic Graph by Best Leaf Identification

Monte Carlo tree search (MCTS) has received considerable interest due to...
10/15/2019

Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions

A plethora of problems in AI, engineering and the sciences are naturally...
12/03/2013

Combining Simulated Annealing and Monte Carlo Tree Search for Expression Simplification

In many applications of computer algebra large expressions must be simpl...
04/23/2020

Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning

Standard planners for sequential decision making (including Monte Carlo ...

Code Repositories

CrazyAra

A Deep Learning UCI-Chess Variant Engine written in C++ & Python :parrot:


view repo

Alpha-Zero-Racing-Kings

Alpha Zero general-purpose reinforcement learning algorithm implemented for the Chess variant Racing Kings.


view repo