DeepAI
Log In Sign Up

Learning to Follow Language Instructions with Compositional Policies

10/09/2021
by   Vanya Cohen, et al.
15

We propose a framework that learns to execute natural language instructions in an environment consisting of goal-reaching tasks that share components of their task descriptions. Our approach leverages the compositionality of both value functions and language, with the aim of reducing the sample complexity of learning novel tasks. First, we train a reinforcement learning agent to learn value functions that can be subsequently composed through a Boolean algebra to solve novel tasks. Second, we fine-tune a seq2seq model pretrained on web-scale corpora to map language to logical expressions that specify the required value function compositions. Evaluating our agent in the BabyAI domain, we observe a decrease of 86 after mastering a single task. Results from ablation studies further indicate that it is the combination of compositional value functions and language representations that allows the agent to quickly generalize to new tasks.

READ FULL TEXT

page 3

page 4

page 6

page 7

01/06/2020

A Boolean Task Algebra for Reinforcement Learning

We propose a framework for defining a Boolean algebra over the space of ...
04/08/2022

Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics

This paper focuses on robotic reinforcement learning with sparse rewards...
02/13/2021

LTL2Action: Generalizing LTL Instructions for Multi-Task RL

We address the problem of teaching a deep reinforcement learning (RL) ag...
02/12/2019

ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

Sparse reward is one of the most challenging problems in reinforcement l...
10/13/2020

Probing for Multilingual Numerical Understanding in Transformer-Based Language Models

Natural language numbers are an example of compositional structures, whe...
05/25/2022

Learning to Query Internet Text for Informing Reinforcement Learning Agents

Generalization to out of distribution tasks in reinforcement learning is...
03/28/2017

A Deep Compositional Framework for Human-like Language Acquisition in Virtual Environment

We tackle a task where an agent learns to navigate in a 2D maze-like env...