Automatic Goal Generation using Dynamical Distance Learning

11/07/2021
by   Bharat Prakash, et al.
0

Reinforcement Learning (RL) agents can learn to solve complex sequential decision making tasks by interacting with the environment. However, sample efficiency remains a major challenge. In the field of multi-goal RL, where agents are required to reach multiple goals to solve complex tasks, improving sample efficiency can be especially challenging. On the other hand, humans or other biological agents learn such tasks in a much more strategic way, following a curriculum where tasks are sampled with increasing difficulty level in order to make gradual and efficient learning progress. In this work, we propose a method for automatic goal generation using a dynamical distance function (DDF) in a self-supervised fashion. DDF is a function which predicts the dynamical distance between any two states within a markov decision process (MDP). With this, we generate a curriculum of goals at the appropriate difficulty level to facilitate efficient learning throughout the training process. We evaluate this approach on several goal-conditioned robotic manipulation and navigation tasks, and show improvements in sample efficiency over a baseline method which only uses random goal sampling.

READ FULL TEXT

page 3

page 4

page 7

research
06/17/2020

Automatic Curriculum Learning through Value Disagreement

Continually solving new, unsolved tasks is the key to learning diverse b...
research
02/18/2020

Generating Automatic Curricula via Self-Supervised Active Domain Randomization

Goal-directed Reinforcement Learning (RL) traditionally considers an age...
research
07/05/2019

Self-supervised Learning of Distance Functions for Goal-Conditioned Reinforcement Learning

Goal-conditioned policies are used in order to break down complex reinfo...
research
02/07/2020

Accelerating Reinforcement Learning for Reaching using Continuous Curriculum Learning

Reinforcement learning has shown great promise in the training of robot ...
research
07/12/2023

Maneuver Decision-Making Through Automatic Curriculum Reinforcement Learning Without Handcrafted Reward functions

Maneuver decision-making is the core of unmanned combat aerial vehicle f...
research
10/15/2018

CURIOUS: Intrinsically Motivated Multi-Task, Multi-Goal Reinforcement Learning

In open-ended and changing environments, agents face a wide range of pot...
research
01/27/2023

Outcome-directed Reinforcement Learning by Uncertainty Temporal Distance-Aware Curriculum Goal Generation

Current reinforcement learning (RL) often suffers when solving a challen...

Please sign up or login with your details

Forgot password? Click here to reset