Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control

by   Frederik Ebert, et al.

Deep reinforcement learning (RL) algorithms can learn complex robotic skills from raw sensory inputs, but have yet to achieve the kind of broad generalization and applicability demonstrated by deep learning methods in supervised domains. We present a deep RL method that is practical for real-world robotics tasks, such as robotic manipulation, and generalizes effectively to never-before-seen tasks and objects. In these settings, ground truth reward signals are typically unavailable, and we therefore propose a self-supervised model-based approach, where a predictive model learns to directly predict the future from raw sensory readings, such as camera images. At test time, we explore three distinct goal specification methods: designated pixels, where a user specifies desired object manipulation tasks by selecting particular pixels in an image and corresponding goal positions, goal images, where the desired goal state is specified with an image, and image classifiers, which define spaces of goal states. Our deep predictive models are trained using data collected autonomously and continuously by a robot interacting with hundreds of objects, without human supervision. We demonstrate that visual MPC can generalize to never-before-seen objects---both rigid and deformable---and solve a range of user-defined object manipulation tasks using the same model.


page 1

page 2

page 6

page 8

page 9

page 11

page 13

page 14


Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning

Prediction is an appealing objective for self-supervised learning of beh...

Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning

Predictive models have been at the core of many robotic systems, from qu...

Self-supervised Learning of Image Embedding for Continuous Control

Operating directly from raw high dimensional sensory inputs like images ...

Learning to Act by Predicting the Future

We present an approach to sensorimotor control in immersive environments...

Learning Robotic Manipulation through Visual Planning and Acting

Planning for robotic manipulation requires reasoning about the changes a...

Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation

Collecting and automatically obtaining reward signals from real robotic ...

Deep Visual Foresight for Planning Robot Motion

A key challenge in scaling up robot learning to many skills and environm...

Please sign up or login with your details

Forgot password? Click here to reset