Learning Latent Graph Dynamics for Deformable Object Manipulation

04/25/2021
by   Xiao Ma, et al.
10

Manipulating deformable objects, such as cloth and ropes, is a long-standing challenge in robotics: their large number of degrees of freedom (DoFs) and complex non-linear dynamics make motion planning extremely difficult. This work aims to learn latent Graph dynamics for DefOrmable Object Manipulation (G-DOOM). To tackle the challenge of many DoFs and complex dynamics, G-DOOM approximates a deformable object as a sparse set of interacting keypoints and learns a graph neural network that captures abstractly the geometry and interaction dynamics of the keypoints. Further, to tackle the perceptual challenge, specifically, object self-occlusion, G-DOOM adds a recurrent neural network to track the keypoints over time and condition their interactions on the history. We then train the resulting recurrent graph dynamics model through contrastive learning in a high-fidelity simulator. For manipulation planning, G-DOOM explicitly reasons about the learned dynamics model through model-predictive control applied at each of the keypoints. We evaluate G-DOOM on a set of challenging cloth and rope manipulation tasks and show that G-DOOM outperforms a state-of-the-art method. Further, although trained entirely on simulation data, G-DOOM transfers directly to a real robot for both cloth and rope manipulation in our experiments.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 5

page 7

page 8

page 12

page 14

03/11/2020

Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

Using visual model-based learning for deformable object manipulation is ...
03/19/2020

Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation

We present a framework for visual action planning of complex manipulatio...
01/29/2020

Learning When to Trust a Dynamics Model for Planning in Reduced State Spaces

When the dynamics of a system are difficult to model and/or time-consumi...
10/24/2021

DiffSRL: Learning Dynamic-aware State Representation for Deformable Object Control with Differentiable Simulator

Dynamic state representation learning is an important task in robot lear...
09/26/2019

Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding

We address the problem of accelerating thin-shell deformable object simu...
03/04/2021

Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects

Capturing scene dynamics and predicting the future scene state is challe...
02/04/2021

Keep it Simple: Data-efficient Learning for Controlling Complex Systems with Simple Models

When manipulating a novel object with complex dynamics, a state represen...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.