ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

02/12/2019
by   Harris Chan, et al.
6

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling the goals. Despite its effectiveness, HER has limited applicability because it lacks a compact and universal goal representation. We present Augmenting experienCe via TeacheR's adviCE (ACTRCE), an efficient reinforcement learning technique that extends the HER framework using natural language as the goal representation. We first analyze the differences among goal representation, and show that ACTRCE can efficiently solve difficult reinforcement learning problems in challenging 3D navigation tasks, whereas HER with non-language goal representation failed to learn. We also show that with language goal representations, the agent can generalize to unseen instructions, and even generalize to instructions with unseen lexicons. We further demonstrate it is crucial to use hindsight advice to solve challenging tasks, and even small amount of advice is sufficient for the agent to achieve good performance.

READ FULL TEXT

page 12

page 19

page 20

page 21

page 22

research
03/05/2019

Using Natural Language for Reward Shaping in Reinforcement Learning

Recent reinforcement learning (RL) approaches have shown strong performa...
research
04/08/2022

Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics

This paper focuses on robotic reinforcement learning with sparse rewards...
research
05/14/2019

Bias-Reduced Hindsight Experience Replay with Virtual Goal Prioritization

Hindsight Experience Replay (HER) is a multi-goal reinforcement learning...
research
02/26/2018

Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research

The purpose of this technical report is two-fold. First of all, it intro...
research
01/31/2019

Visual Hindsight Experience Replay

Reinforcement Learning algorithms typically require millions of environm...
research
01/06/2020

Learning Reusable Options for Multi-Task Reinforcement Learning

Reinforcement learning (RL) has become an increasingly active area of re...
research
10/09/2021

Learning to Follow Language Instructions with Compositional Policies

We propose a framework that learns to execute natural language instructi...

Please sign up or login with your details

Forgot password? Click here to reset