Monte-Carlo Tree Search as Regularized Policy Optimization

07/24/2020
by   Jean-Bastien Grill, et al.
5

The combination of Monte-Carlo tree search (MCTS) with deep reinforcement learning has led to significant advances in artificial intelligence. However, AlphaZero, the current state-of-the-art MCTS algorithm, still relies on handcrafted heuristics that are only partially understood. In this paper, we show that AlphaZero's search heuristics, along with other common ones such as UCT, are an approximation to the solution of a specific regularized policy optimization problem. With this insight, we propose a variant of AlphaZero which uses the exact solution to this policy optimization problem, and show experimentally that it reliably outperforms the original algorithm in multiple domains.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/14/2020

Solve Traveling Salesman Problem by Monte Carlo Tree Search and Deep Neural Network

We present a self-learning approach that combines deep reinforcement lea...
research
05/22/2020

Single-Agent Optimization Through Policy Iteration Using Monte-Carlo Tree Search

The combination of Monte-Carlo Tree Search (MCTS) and deep reinforcement...
research
09/09/2015

A Topological Approach to Meta-heuristics: Analytical Results on the BFS vs. DFS Algorithm Selection Problem

Search is a central problem in artificial intelligence, and BFS and DFS ...
research
06/01/2019

Automated Machine Learning with Monte-Carlo Tree Search (Extended Version)

The AutoML task consists of selecting the proper algorithm in a machine ...
research
06/17/2022

Efficient Object Manipulation Planning with Monte Carlo Tree Search

This paper presents an efficient approach to object manipulation plannin...
research
07/28/2021

Monte Carlo Tree Search for high precision manufacturing

Monte Carlo Tree Search (MCTS) has shown its strength for a lot of deter...
research
11/15/2021

Learning Robust Scheduling with Search and Attention

Allocating physical layer resources to users based on channel quality, b...

Please sign up or login with your details

Forgot password? Click here to reset