Density-based Curriculum for Multi-goal Reinforcement Learning with Sparse Rewards

09/18/2021
by   Deyu Yang, et al.
0

Multi-goal reinforcement learning (RL) aims to qualify the agent to accomplish multi-goal tasks, which is of great importance in learning scalable robotic manipulation skills. However, reward engineering always requires strenuous efforts in multi-goal RL. Moreover, it will introduce inevitable bias causing the suboptimality of the final policy. The sparse reward provides a simple yet efficient way to overcome such limits. Nevertheless, it harms the exploration efficiency and even hinders the policy from convergence. In this paper, we propose a density-based curriculum learning method for efficient exploration with sparse rewards and better generalization to desired goal distribution. Intuitively, our method encourages the robot to gradually broaden the frontier of its ability along the directions to cover the entire desired goal space as much and quickly as possible. To further improve data efficiency and generality, we augment the goals and transitions within the allowed region during training. Finally, We evaluate our method on diversified variants of benchmark manipulation tasks that are challenging for existing methods. Empirical results show that our method outperforms the state-of-the-art baselines in terms of both data efficiency and success rate.

READ FULL TEXT

page 1

page 5

page 6

research
10/28/2022

Goal Exploration Augmentation via Pre-trained Skills for Sparse-Reward Long-Horizon Goal-Conditioned Reinforcement Learning

Reinforcement learning (RL) often struggles to accomplish a sparse-rewar...
research
07/04/2018

Region Growing Curriculum Generation for Reinforcement Learning

Learning a policy capable of moving an agent between any two states in t...
research
08/04/2021

Parallelized Reverse Curriculum Generation

For reinforcement learning (RL), it is challenging for an agent to maste...
research
01/27/2023

Outcome-directed Reinforcement Learning by Uncertainty Temporal Distance-Aware Curriculum Goal Generation

Current reinforcement learning (RL) often suffers when solving a challen...
research
10/25/2021

Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning

Learning in a multi-target environment without prior knowledge about the...
research
08/01/2022

Relay Hindsight Experience Replay: Continual Reinforcement Learning for Robot Manipulation Tasks with Sparse Rewards

Learning with sparse rewards is usually inefficient in Reinforcement Lea...
research
09/23/2022

Multi-Agent Exploration of an Unknown Sparse Landmark Complex via Deep Reinforcement Learning

In recent years Landmark Complexes have been successfully employed for l...

Please sign up or login with your details

Forgot password? Click here to reset