Learning to Smooth and Fold Real Fabric Using Dense Object Descriptors Trained on Synthetic Color Images

by   Aditya Ganapathi, et al.

Robotic fabric manipulation is challenging due to the infinite dimensional configuration space and complex dynamics. In this paper, we learn visual representations of deformable fabric by training dense object descriptors that capture correspondences across images of fabric in various configurations. The learned descriptors capture higher level geometric structure, facilitating design of explainable policies. We demonstrate that the learned representation facilitates multistep fabric smoothing and folding tasks on two real physical systems, the da Vinci surgical robot and the ABB YuMi given high level demonstrations from a supervisor. The system achieves a 78.8 success rate across six fabric manipulation tasks. See https://tinyurl.com/fabric-descriptors for supplementary material and videos.



page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8


Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data

Robotic manipulation of deformable 1D objects such as ropes, cables, and...

Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies

Robot manipulation for untangling 1D deformable structures such as ropes...

Untangling Dense Knots by Learning Task-Relevant Keypoints

Untangling ropes, wires, and cables is a challenging task for robots due...

Deep Imitation Learning of Sequential Fabric Smoothing Policies

Sequential pulling policies to flatten and smooth fabrics have applicati...

VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulation

Robotic fabric manipulation has applications in cloth and cable manageme...

MMGSD: Multi-Modal Gaussian Shape Descriptors for Correspondence Matching in 1D and 2D Deformable Objects

We explore learning pixelwise correspondences between images of deformab...

VisuoSpatial Foresight for Physical Sequential Fabric Manipulation

Robotic fabric manipulation has applications in home robotics, textiles,...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.