MMGSD: Multi-Modal Gaussian Shape Descriptors for Correspondence Matching in 1D and 2D Deformable Objects

by   Aditya Ganapathi, et al.

We explore learning pixelwise correspondences between images of deformable objects in different configurations. Traditional correspondence matching approaches such as SIFT, SURF, and ORB can fail to provide sufficient contextual information for fine-grained manipulation. We propose Multi-Modal Gaussian Shape Descriptor (MMGSD), a new visual representation of deformable objects which extends ideas from dense object descriptors to predict all symmetric correspondences between different object configurations. MMGSD is learned in a self-supervised manner from synthetic data and produces correspondence heatmaps with measurable uncertainty. In simulation, experiments suggest that MMGSD can achieve an RMSE of 32.4 and 31.3 for square cloth and braided synthetic nylon rope respectively. The results demonstrate an average of 47.7 symmetric pixel-wise contrastive loss (SPCL), as opposed to MMGSD which enforces distributional continuity.



page 1

page 2


DASC: Robust Dense Descriptor for Multi-modal and Multi-spectral Correspondence Estimation

Establishing dense correspondences between multiple images is a fundamen...

Bayesian Inference of Bijective Non-Rigid Shape Correspondence

Many algorithms for the computation of correspondences between deformabl...

Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data

Robotic manipulation of deformable 1D objects such as ropes, cables, and...

Learning to Smooth and Fold Real Fabric Using Dense Object Descriptors Trained on Synthetic Color Images

Robotic fabric manipulation is challenging due to the infinite dimension...

NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields

Thin, reflective objects such as forks and whisks are common in our dail...

Dense 3D Face Correspondence

We present an algorithm that automatically establishes dense corresponde...

Self-Supervised Correspondence in Visuomotor Policy Learning

In this paper we explore using self-supervised correspondence for improv...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.