Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning

09/10/2020
by   Lucas Manuelli, et al.
0

Predictive models have been at the core of many robotic systems, from quadrotors to walking robots. However, it has been challenging to develop and apply such models to practical robotic manipulation due to high-dimensional sensory observations such as images. Previous approaches to learning models in the context of robotic manipulation have either learned whole image dynamics or used autoencoders to learn dynamics in a low-dimensional latent state. In this work, we introduce model-based prediction with self-supervised visual correspondence learning, and show that not only is this indeed possible, but demonstrate that these types of predictive models show compelling performance improvements over alternative methods for vision-based RL with autoencoder-type vision training. Through simulation experiments, we demonstrate that our models provide better generalization precision, particularly in 3D scenes, scenes involving occlusion, and in category-generalization. Additionally, we validate that our method effectively transfers to the real world through hardware experiments. Videos and supplementary materials available at https://sites.google.com/view/keypointsintothefuture

READ FULL TEXT

page 2

page 5

page 6

page 15

page 16

06/28/2022

Masked World Models for Visual Control

Visual model-based reinforcement learning (RL) has the potential to enab...
11/13/2020

ROLL: Visual Self-Supervised Reinforcement Learning with Object Reasoning

Current image-based reinforcement learning (RL) algorithms typically ope...
12/03/2018

Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control

Deep reinforcement learning (RL) algorithms can learn complex robotic sk...
05/21/2021

Learning Visible Connectivity Dynamics for Cloth Smoothing

Robotic manipulation of cloth remains challenging for robotics due to th...
12/03/2019

Integrating Motion into Vision Models for Better Visual Prediction

We demonstrate an improved vision system that learns a model of its envi...
09/12/2019

Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation

Video prediction models combined with planning algorithms have shown pro...
12/12/2020

Sampling Training Data for Continual Learning Between Robots and the Cloud

Today's robotic fleets are increasingly measuring high-volume video and ...