Goal-Conditioned Supervised Learning with Sub-Goal Prediction

05/17/2023
by   Tom Jurgenson, et al.
0

Recently, a simple yet effective algorithm – goal-conditioned supervised-learning (GCSL) – was proposed to tackle goal-conditioned reinforcement-learning. GCSL is based on the principle of hindsight learning: by observing states visited in previously executed trajectories and treating them as attained goals, GCSL learns the corresponding actions via supervised learning. However, GCSL only learns a goal-conditioned policy, discarding other information in the process. Our insight is that the same hindsight principle can be used to learn to predict goal-conditioned sub-goals from the same trajectory. Based on this idea, we propose Trajectory Iterative Learner (TraIL), an extension of GCSL that further exploits the information in a trajectory, and uses it for learning to predict both actions and sub-goals. We investigate the settings in which TraIL can make better use of the data, and discover that for several popular problem settings, replacing real goals in GCSL with predicted TraIL sub-goals allows the agent to reach a greater set of goal states using the exact same data as GCSL, thereby improving its overall performance.

READ FULL TEXT

page 6

page 7

page 9

research
11/03/2019

Learning from Trajectories via Subgoal Discovery

Learning to solve complex goal-oriented tasks with sparse terminal-only ...
research
04/11/2021

Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning

It is of significance for an agent to learn a widely applicable and gene...
research
06/12/2019

Sub-Goal Trees -- a Framework for Goal-Directed Trajectory Prediction and Optimization

Many AI problems, in robotics and other domains, are goal-directed, esse...
research
02/27/2020

Sub-Goal Trees – a Framework for Goal-Based Reinforcement Learning

Many AI problems, in robotics and other domains, are goal-based, essenti...
research
12/22/2016

First-Person Activity Forecasting with Online Inverse Reinforcement Learning

We address the problem of incrementally modeling and forecasting long-te...
research
03/26/2023

Learning Generative Models with Goal-conditioned Reinforcement Learning

We present a novel, alternative framework for learning generative models...
research
05/20/2022

A Fully Controllable Agent in the Path Planning using Goal-Conditioned Reinforcement Learning

The aim of path planning is to reach the goal from starting point by sea...

Please sign up or login with your details

Forgot password? Click here to reset