DeepAI AI Chat
Log In Sign Up

Revisiting the Roles of "Text" in Text Games

10/15/2022
by   Yi Gu, et al.
Princeton University
Tencent
University of California, San Diego
9

Text games present opportunities for natural language understanding (NLU) methods to tackle reinforcement learning (RL) challenges. However, recent work has questioned the necessity of NLU by showing random text hashes could perform decently. In this paper, we pursue a fine-grained investigation into the roles of text in the face of different RL challenges, and reconcile that semantic and non-semantic language representations could be complementary rather than contrasting. Concretely, we propose a simple scheme to extract relevant contextual information into an approximate state hash as extra input for an RNN-based text agent. Such a lightweight plug-in achieves competitive performance with state-of-the-art text agents using advanced NLU techniques such as knowledge graph and passage retrieval, suggesting non-NLU methods might suffice to tackle the challenge of partial observability. However, if we remove RNN encoders and use approximate or even ground-truth state hash alone, the model performs miserably, which confirms the importance of semantic function approximation to tackle the challenge of combinatorially large observation and action spaces. Our findings and analysis provide new insights for designing better text game task setups and agents.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/23/2020

Graph Constrained Reinforcement Learning for Natural Language Action Spaces

Interactive Fiction games are text-based simulations in which an agent i...
07/18/2021

Pre-trained Language Models as Prior Knowledge for Playing Text-based Games

Recently, text world games have been proposed to enable artificial agent...
10/22/2020

Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games

We study reinforcement learning (RL) for text-based games, which are int...
03/25/2021

Reading and Acting while Blindfolded: The Need for Semantics in Text Game Agents

Text-based games simulate worlds and interact with players using natural...
06/10/2019

A Survey of Reinforcement Learning Informed by Natural Language

To be successful in real-world tasks, Reinforcement Learning (RL) needs ...
09/24/2020

Bootstrapped Q-learning with Context Relevant Observation Pruning to Generalize in Text-based Games

We show that Reinforcement Learning (RL) methods for solving Text-Based ...
08/01/2022

TextWorldExpress: Simulating Text Games at One Million Steps Per Second

Text-based games offer a challenging test bed to evaluate virtual agents...