Learning to Win by Reading Manuals in a Monte-Carlo Framework

01/18/2014
by   S. R. K. Branavan, et al.
0

Domain knowledge is crucial for effective performance in autonomous control systems. Typically, human effort is required to encode this knowledge into a control algorithm. In this paper, we present an approach to language grounding which automatically interprets text in the context of a complex control application, such as a game, and uses domain knowledge extracted from the text to improve control performance. Both text analysis and control strategies are learned jointly using only a feedback signal inherent to the application. To effectively leverage textual information, our method automatically extracts the text segment most relevant to the current game state, and labels it with a task-centric predicate structure. This labeled text is then used to bias an action selection policy for the game, guiding it towards promising regions of the action space. We encode our model for text analysis and game playing in a multi-layer neural network, representing linguistic decisions via latent variables in the hidden layers, and game action quality via the output layer. Operating within the Monte-Carlo Search framework, we estimate model parameters using feedback from simulated games. We apply our approach to the complex strategy game Civilization II using the official game manual as the text guide. Our results show that a linguistically-informed game-playing agent significantly outperforms its language-unaware counterpart, yielding a 34 absolute improvement and winning over 65 built-in AI of Civilization.

READ FULL TEXT

page 19

page 37

research
01/23/2021

Deep Learning for General Game Playing with Ludii and Polygames

Combinations of Monte-Carlo tree search and Deep Neural Networks, traine...
research
07/12/2018

Monte Carlo Methods for the Game Kingdomino

Kingdomino is introduced as an interesting game for studying game playin...
research
11/07/2018

Baselines for Reinforcement Learning in Text Games

The ability to learn optimal control policies in systems where action sp...
research
12/18/2018

Monte Carlo Continual Resolving for Online Strategy Computation in Imperfect Information Games

Online game playing algorithms produce high-quality strategies with a fr...
research
08/19/2019

Transfer in Deep Reinforcement Learning using Knowledge Graphs

Text adventure games, in which players must make sense of the world thro...
research
05/30/2022

Elastic Monte Carlo Tree Search with State Abstraction for Strategy Game Playing

Strategy video games challenge AI agents with their combinatorial search...
research
12/14/2021

Split Moves for Monte-Carlo Tree Search

In many games, moves consist of several decisions made by the player. Th...

Please sign up or login with your details

Forgot password? Click here to reset