ACDER: Augmented Curiosity-Driven Experience Replay

11/16/2020
by   Boyao Li, et al.
0

Exploration in environments with sparse feedback remains a challenging research problem in reinforcement learning (RL). When the RL agent explores the environment randomly, it results in low exploration efficiency, especially in robotic manipulation tasks with high dimensional continuous state and action space. In this paper, we propose a novel method, called Augmented Curiosity-Driven Experience Replay (ACDER), which leverages (i) a new goal-oriented curiosity-driven exploration to encourage the agent to pursue novel and task-relevant states more purposefully and (ii) the dynamic initial states selection as an automatic exploratory curriculum to further improve the sample-efficiency. Our approach complements Hindsight Experience Replay (HER) by introducing a new way to pursue valuable states. Experiments conducted on four challenging robotic manipulation tasks with binary rewards, including Reach, Push, Pick Place and Multi-step Push. The empirical results show that our proposed method significantly outperforms existing methods in the first three basic tasks and also achieves satisfactory performance in multi-step robotic task learning.

READ FULL TEXT

page 5

page 6

research
02/01/2019

Competitive Experience Replay

Deep learning has achieved remarkable successes in solving challenging r...
research
07/05/2017

Hindsight Experience Replay

Dealing with sparse rewards is one of the biggest challenges in Reinforc...
research
02/20/2019

Curiosity-Driven Experience Prioritization via Density Estimation

In Reinforcement Learning (RL), an agent explores the environment and co...
research
10/02/2018

Energy-Based Hindsight Experience Prioritization

In Hindsight Experience Replay (HER), a reinforcement learning agent is ...
research
08/01/2022

Relay Hindsight Experience Replay: Continual Reinforcement Learning for Robot Manipulation Tasks with Sparse Rewards

Learning with sparse rewards is usually inefficient in Reinforcement Lea...
research
02/26/2018

Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research

The purpose of this technical report is two-fold. First of all, it intro...
research
04/21/2019

Generative Exploration and Exploitation

Sparse reward is one of the biggest challenges in reinforcement learning...

Please sign up or login with your details

Forgot password? Click here to reset