Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics

04/08/2022
by   Frank Röder, et al.
0

This paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. An open problem is the sample-inefficiency that stems from the compositionality of natural language, and from the grounding of language in sensory data and actions. We address these issues with three contributions. We first present a mechanism for hindsight instruction replay utilizing expert feedback. Second, we propose a seq2seq model to generate linguistic hindsight instructions. Finally, we present a novel class of language-focused learning tasks. We show that hindsight instructions improve the learning performance, as expected. In addition, we also provide an unexpected result: We show that the learning performance of our agent can be improved by one third if, in a sense, the agent learns to talk to itself in a self-supervised manner. We achieve this by learning to generate linguistic instructions that would have been appropriate as a natural language goal for an originally unintended behavior. Our results indicate that the performance gain increases with the task-complexity.

READ FULL TEXT

page 1

page 4

research
06/03/2021

Grounding Complex Navigational Instructions Using Scene Graphs

Training a reinforcement learning agent to carry out natural language in...
research
06/22/2017

Gated-Attention Architectures for Task-Oriented Language Grounding

To perform tasks specified by natural language instructions, autonomous ...
research
02/12/2019

ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

Sparse reward is one of the most challenging problems in reinforcement l...
research
10/09/2021

Learning to Follow Language Instructions with Compositional Policies

We propose a framework that learns to execute natural language instructi...
research
06/20/2017

Grounded Language Learning in a Simulated 3D World

We are increasingly surrounded by artificially intelligent technology th...
research
10/27/2019

Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order

In this work, we analyze the performance of general deep reinforcement l...
research
11/09/2015

A disembodied developmental robotic agent called Samu Bátfai

The agent program, called Samu, is an experiment to build a disembodied ...

Please sign up or login with your details

Forgot password? Click here to reset