Learning to Smooth and Fold Real Fabric Using Dense Object Descriptors Trained on Synthetic Color Images

03/28/2020
by   Aditya Ganapathi, et al.
15

Robotic fabric manipulation is challenging due to the infinite dimensional configuration space and complex dynamics. In this paper, we learn visual representations of deformable fabric by training dense object descriptors that capture correspondences across images of fabric in various configurations. The learned descriptors capture higher level geometric structure, facilitating design of explainable policies. We demonstrate that the learned representation facilitates multistep fabric smoothing and folding tasks on two real physical systems, the da Vinci surgical robot and the ABB YuMi given high level demonstrations from a supervisor. The system achieves a 78.8 success rate across six fabric manipulation tasks. See https://tinyurl.com/fabric-descriptors for supplementary material and videos.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8

research
03/03/2020

Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data

Robotic manipulation of deformable 1D objects such as ropes, cables, and...
research
06/29/2021

Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies

Robot manipulation for untangling 1D deformable structures such as ropes...
research
11/10/2020

Untangling Dense Knots by Learning Task-Relevant Keypoints

Untangling ropes, wires, and cables is a challenging task for robots due...
research
09/23/2019

Deep Imitation Learning of Sequential Fabric Smoothing Policies

Sequential pulling policies to flatten and smooth fabrics have applicati...
research
10/09/2020

MMGSD: Multi-Modal Gaussian Shape Descriptors for Correspondence Matching in 1D and 2D Deformable Objects

We explore learning pixelwise correspondences between images of deformab...
research
04/18/2023

Learning Sim-to-Real Dense Object Descriptors for Robotic Manipulation

It is crucial to address the following issues for ubiquitous robotics ma...
research
03/20/2023

Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation

Understanding and manipulating deformable objects (e.g., ropes and fabri...

Please sign up or login with your details

Forgot password? Click here to reset