Improvements on Hindsight Learning

09/16/2018
by   Ameet Deshpande, et al.
0

Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising way to train an agent to perform goal-directed tasks is to use Hindsight Learning approaches. In these approaches, even when an agent fails to reach the desired goal, the agent learns to reach the goal it achieved instead. Doing this over multiple trajectories while generalizing the policy learned from the achieved goals, the agent learns a goal conditioned policy to reach any goal. One such approach is Hindsight Experience replay which uses an off-policy Reinforcement Learning algorithm to learn a goal conditioned policy. In this approach, a replay of the past transitions happens in a uniformly random fashion. Another approach is to use a Hindsight version of the policy gradients to directly learn a policy. In this work, we discuss different ways to replay past transitions to improve learning in hindsight experience replay focusing on prioritized variants in particular. Also, we implement the Hindsight Policy gradient methods to robotic tasks.

READ FULL TEXT
research
05/21/2019

Maximum Entropy-Regularized Multi-Goal Reinforcement Learning

In Multi-Goal Reinforcement Learning, an agent learns to achieve multipl...
research
08/17/2021

Diversity-based Trajectory and Goal Selection with Hindsight Experience Replay

Hindsight experience replay (HER) is a goal relabelling technique typica...
research
02/15/2023

Prioritized offline Goal-swapping Experience Replay

In goal-conditioned offline reinforcement learning, an agent learns from...
research
11/27/2018

Prioritizing Starting States for Reinforcement Learning

Online, off-policy reinforcement learning algorithms are able to use an ...
research
05/14/2019

Bias-Reduced Hindsight Experience Replay with Virtual Goal Prioritization

Hindsight Experience Replay (HER) is a multi-goal reinforcement learning...
research
01/31/2019

Visual Hindsight Experience Replay

Reinforcement Learning algorithms typically require millions of environm...
research
09/30/2021

Real Robot Challenge using Deep Reinforcement Learning

This paper details our winning submission to Phase 1 of the 2021 Real Ro...

Please sign up or login with your details

Forgot password? Click here to reset