Learning to Search with MCTSnets

02/13/2018
by   Arthur Guez, et al.
0

Planning problems are among the most important and well-studied problems in artificial intelligence. They are most typically solved by tree search algorithms that simulate ahead into the future, evaluate future states, and back-up those evaluations to the root of a search tree. Among these algorithms, Monte-Carlo tree search (MCTS) is one of the most general, powerful and widely used. A typical implementation of MCTS uses cleverly designed rules, optimized to the particular characteristics of the domain. These rules control where the simulation traverses, what to evaluate in the states that are reached, and how to back-up those evaluations. In this paper we instead learn where, what and how to search. Our architecture, which we call an MCTSnet, incorporates simulation-based search inside a neural network, by expanding, evaluating and backing-up a vector embedding. The parameters of the network are trained end-to-end using gradient-based optimisation. When applied to small searches in the well known planning problem Sokoban, the learned search algorithm significantly outperformed MCTS baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/07/2019

Policy Gradient Search: Online Planning and Expert Iteration without Search Trees

Monte Carlo Tree Search (MCTS) algorithms perform simulation-based searc...
research
04/09/2021

Batch Monte Carlo Tree Search

Making inferences with a deep neural network on a batch of states is muc...
research
04/01/2019

Tree Search Network for Sparse Regression

We consider the classical sparse regression problem of recovering a spar...
research
10/21/2015

GLASSES: Relieving The Myopia Of Bayesian Optimisation

We present GLASSES: Global optimisation with Look-Ahead through Stochast...
research
09/27/2022

Efficient Non-Parametric Optimizer Search for Diverse Tasks

Efficient and automated design of optimizers plays a crucial role in ful...
research
02/11/2020

Static and Dynamic Values of Computation in MCTS

Monte-Carlo Tree Search (MCTS) is one of the most-widely used methods fo...
research
12/29/2018

StarAlgo: A Squad Movement Planning Library for StarCraft using Monte Carlo Tree Search and Negamax

Real-Time Strategy (RTS) games have recently become a popular testbed fo...

Please sign up or login with your details

Forgot password? Click here to reset