Object-centric Forward Modeling for Model Predictive Control

10/08/2019
by   Yufei Ye, et al.
7

We present an approach to learn an object-centric forward model, and show that this allows us to plan for sequences of actions to achieve distant desired goals. We propose to model a scene as a collection of objects, each with an explicit spatial location and implicit visual feature, and learn to model the effects of actions using random interaction data. Our model allows capturing the robot-object and object-object interactions, and leads to more sample-efficient and accurate predictions. We show that this learned model can be leveraged to search for action sequences that lead to desired goal configurations, and that in conjunction with a learned correction module, this allows for robust closed loop execution. We present experiments both in simulation and the real world, and show that our approach improves over alternate implicit or pixel-space forward models. Please see our project page (https://judyye.github.io/ocmpc/) for result videos.

READ FULL TEXT

page 6

page 7

page 8

page 13

research
09/08/2021

SORNet: Spatial Object-Centric Representations for Sequential Manipulation

Sequential manipulation tasks require a robot to perceive the state of a...
research
04/27/2023

Compositional 3D Human-Object Neural Animation

Human-object interactions (HOIs) are crucial for human-centric scene und...
research
03/07/2021

Robotic Visuomotor Control with Unsupervised Forward Model Learned from Videos

Learning an accurate model of the environment is essential for model-bas...
research
08/02/2020

Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interaction

A key challenge for an agent learning to interact with the world is to r...
research
10/09/2021

Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly

Studies in robot teleoperation have been centered around action specific...
research
05/26/2016

Low-Cost Scene Modeling using a Density Function Improves Segmentation Performance

We propose a low cost and effective way to combine a free simulation sof...
research
03/30/2019

Learning Semantic Embedding Spaces for Slicing Vegetables

In this work, we present an interaction-based approach to learn semantic...

Please sign up or login with your details

Forgot password? Click here to reset