Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation

09/12/2019
by   Suraj Nair, et al.
4

Video prediction models combined with planning algorithms have shown promise in enabling robots to learn to perform many vision-based tasks through only self-supervision, reaching novel goals in cluttered scenes with unseen objects. However, due to the compounding uncertainty in long horizon video prediction and poor scalability of sampling-based planning optimizers, one significant limitation of these approaches is the ability to plan over long horizons to reach distant goals. To that end, we propose a framework for subgoal generation and planning, hierarchical visual foresight (HVF), which generates subgoal images conditioned on a goal image, and uses them for planning. The subgoal images are directly optimized to decompose the task into easy to plan segments, and as a result, we observe that the method naturally identifies semantically meaningful states as subgoals. Across three out of four simulated vision-based manipulation tasks, we find that our method achieves nearly a 200 improvement over planning without subgoals and model-free RL approaches. Further, our experiments illustrate that our approach extends to real, cluttered visual scenes. Project page: https://sites.google.com/stanford.edu/hvf

READ FULL TEXT

page 4

page 7

page 8

page 16

research
09/28/2019

Regression Planning Networks

Recent learning-to-plan methods have shown promising results on planning...
research
10/02/2018

Time Reversal as Self-Supervision

A longstanding challenge in robot learning for manipulation tasks has be...
research
06/23/2020

Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors

The ability to predict and plan into the future is fundamental for agent...
research
09/10/2020

Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning

Predictive models have been at the core of many robotic systems, from qu...
research
07/14/2020

Goal-Aware Prediction: Learning to Model What Matters

Learned dynamics models combined with both planning and policy learning ...
research
09/15/2023

Compositional Foundation Models for Hierarchical Planning

To make effective decisions in novel environments with long-horizon goal...
research
10/18/2021

Discovering and Achieving Goals via World Models

How can artificial agents learn to solve many diverse tasks in complex v...

Please sign up or login with your details

Forgot password? Click here to reset