Reinforcement Learning of Implicit and Explicit Control Flow in Instructions

02/25/2021
by   Ethan A. Brooks, et al.
21

Learning to flexibly follow task instructions in dynamic environments poses interesting challenges for reinforcement learning agents. We focus here on the problem of learning control flow that deviates from a strict step-by-step execution of instructions – that is, control flow that may skip forward over parts of the instructions or return backward to previously completed or skipped steps. Demand for such flexible control arises in two fundamental ways: explicitly when control is specified in the instructions themselves (such as conditional branching and looping) and implicitly when stochastic environment dynamics require re-completion of instructions whose effects have been perturbed, or opportunistic skipping of instructions whose effects are already present. We formulate an attention-based architecture that meets these challenges by learning, from task reward only, to flexibly attend to and condition behavior on an internal encoding of the instructions. We test the architecture's ability to learn both explicit and implicit control in two illustrative domains – one inspired by Minecraft and the other by StarCraft – and show that the architecture exhibits zero-shot generalization to novel instructions of length greater than those in a training set, at a performance level unmatched by two baseline recurrent architectures and one ablation architecture.

READ FULL TEXT
research
06/15/2017

Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning

As a step towards developing zero-shot task generalization capabilities ...
research
06/05/2018

Learning to Follow Language Instructions with Adversarial Reward Induction

Recent work has shown that deep reinforcement-learning agents can learn ...
research
09/03/2013

Understanding Evolutionary Potential in Virtual CPU Instruction Set Architectures

We investigate fundamental decisions in the design of instruction set ar...
research
09/08/2023

Compositional Learning of Visually-Grounded Concepts Using Reinforcement

Deep reinforcement learning agents need to be trained over millions of e...
research
09/11/2022

Meta-Reinforcement Learning via Language Instructions

Although deep reinforcement learning has recently been very successful a...
research
05/25/2018

Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation

We propose a learning approach for mapping context-dependent sequential ...
research
07/06/2023

Towards Efficient Control Flow Handling in Spatial Architecture via Architecting the Control Flow Plane

Spatial architecture is a high-performance architecture that uses contro...

Please sign up or login with your details

Forgot password? Click here to reset