Beyond Monte Carlo Tree Search: Playing Go with Deep Alternative Neural Network and Long-Term Evaluation

06/13/2017
by   Jinzhuo Wang, et al.
0

Monte Carlo tree search (MCTS) is extremely popular in computer Go which determines each action by enormous simulations in a broad and deep search tree. However, human experts select most actions by pattern analysis and careful evaluation rather than brute search of millions of future nteractions. In this paper, we propose a computer Go system that follows experts way of thinking and playing. Our system consists of two parts. The first part is a novel deep alternative neural network (DANN) used to generate candidates of next move. Compared with existing deep convolutional neural network (DCNN), DANN inserts recurrent layer after each convolutional layer and stacks them in an alternative manner. We show such setting can preserve more contexts of local features and its evolutions which are beneficial for move prediction. The second part is a long-term evaluation (LTE) module used to provide a reliable evaluation of candidates rather than a single probability from move predictor. This is consistent with human experts nature of playing since they can foresee tens of steps to give an accurate estimation of candidates. In our system, for each candidate, LTE calculates a cumulative reward after several future interactions when local variations are settled. Combining criteria from the two parts, our system determines the optimal choice of next move. For more comprehensive experiments, we introduce a new professional Go dataset (PGD), consisting of 253233 professional records. Experiments on GoGoD and PGD datasets show the DANN can substantially improve performance of move prediction over pure DCNN. When combining LTE, our system outperforms most relevant approaches and open engines based on MCTS.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/10/2015

Convolutional Monte Carlo Rollouts in Go

In this work, we present a MCTS-based Go-playing program which uses conv...
research
11/19/2015

Better Computer Go Player with Neural Network and Long-term Prediction

Competing with top human players in the ancient game of Go has been a lo...
research
12/20/2014

Move Evaluation in Go Using Deep Convolutional Neural Networks

The game of Go is more challenging than other board games, due to the di...
research
12/10/2014

Teaching Deep Convolutional Neural Networks to Play Go

Mastering the game of Go has remained a long standing challenge to the f...
research
12/14/2016

Collaborative creativity with Monte-Carlo Tree Search and Convolutional Neural Networks

We investigate a human-machine collaborative drawing environment in whic...
research
05/31/2021

Supervised learning and tree search for real-time storage allocation in Robotic Mobile Fulfillment Systems

A Robotic Mobile Fulfillment System is a robotised parts-to-picker syste...

Please sign up or login with your details

Forgot password? Click here to reset