Learning to Follow Language Instructions with Compositional Policies

10/09/2021
by   Vanya Cohen, et al.
15

We propose a framework that learns to execute natural language instructions in an environment consisting of goal-reaching tasks that share components of their task descriptions. Our approach leverages the compositionality of both value functions and language, with the aim of reducing the sample complexity of learning novel tasks. First, we train a reinforcement learning agent to learn value functions that can be subsequently composed through a Boolean algebra to solve novel tasks. Second, we fine-tune a seq2seq model pretrained on web-scale corpora to map language to logical expressions that specify the required value function compositions. Evaluating our agent in the BabyAI domain, we observe a decrease of 86 after mastering a single task. Results from ablation studies further indicate that it is the combination of compositional value functions and language representations that allows the agent to quickly generalize to new tasks.

READ FULL TEXT

page 3

page 4

page 6

page 7

research
01/06/2020

A Boolean Task Algebra for Reinforcement Learning

We propose a framework for defining a Boolean algebra over the space of ...
research
04/08/2022

Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics

This paper focuses on robotic reinforcement learning with sparse rewards...
research
02/13/2021

LTL2Action: Generalizing LTL Instructions for Multi-Task RL

We address the problem of teaching a deep reinforcement learning (RL) ag...
research
09/08/2023

Compositional Learning of Visually-Grounded Concepts Using Reinforcement

Deep reinforcement learning agents need to be trained over millions of e...
research
02/12/2019

ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

Sparse reward is one of the most challenging problems in reinforcement l...
research
10/13/2020

Probing for Multilingual Numerical Understanding in Transformer-Based Language Models

Natural language numbers are an example of compositional structures, whe...
research
03/28/2017

A Deep Compositional Framework for Human-like Language Acquisition in Virtual Environment

We tackle a task where an agent learns to navigate in a 2D maze-like env...

Please sign up or login with your details

Forgot password? Click here to reset