DeepAI
Log In Sign Up

Learning Visible Connectivity Dynamics for Cloth Smoothing

05/21/2021
by   Xingyu Lin, et al.
4

Robotic manipulation of cloth remains challenging for robotics due to the complex dynamics of the cloth, lack of a low-dimensional state representation, and self-occlusions. In contrast to previous model-based approaches that learn a pixel-based dynamics model or a compressed latent vector dynamics, we propose to learn a particle-based dynamics model from a partial point cloud observation. To overcome the challenges of partial observability, we infer which visible points are connected on the underlying cloth mesh. We then learn a dynamics model over this visible connectivity graph. Compared to previous learning-based approaches, our model poses strong inductive bias with its particle based representation for learning the underlying cloth physics; it is invariant to visual features; and the predictions can be more easily visualized. We show that our method greatly outperforms previous state-of-the-art model-based and model-free reinforcement learning methods in simulation. Furthermore, we demonstrate zero-shot sim-to-real transfer where we deploy the model trained in simulation on a Franka arm and show that the model can successfully smooth different types of cloth from crumpled configurations. Videos can be found on our project website.

READ FULL TEXT

page 1

page 3

page 7

page 8

page 9

page 12

page 15

09/10/2020

Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning

Predictive models have been at the core of many robotic systems, from qu...
01/26/2019

Neural Networks Predict Fluid Dynamics Solutions from Tiny Datasets

In computational fluid dynamics, it often takes days or weeks to simulat...
08/28/2018

SOLAR: Deep Structured Latent Representations for Model-Based Reinforcement Learning

Model-based reinforcement learning (RL) methods can be broadly categoriz...
09/07/2022

Concept-modulated model-based offline reinforcement learning for rapid generalization

The robustness of any machine learning solution is fundamentally bound b...
09/29/2022

Contrastive Unsupervised Learning of World Model with Invariant Causal Features

In this paper we present a world model, which learns causal features usi...
07/18/2021

SENSORIMOTOR GRAPH: Action-Conditioned Graph Neural Network for Learning Robotic Soft Hand Dynamics

Soft robotics is a thriving branch of robotics which takes inspiration f...