Learning Dense Correspondence via 3D-guided Cycle Consistency

04/18/2016
by   Tinghui Zhou, et al.
0

Discriminative deep learning approaches have shown impressive results for problems where human-labeled ground truth is plentiful, but what about tasks where labels are difficult or impossible to obtain? This paper tackles one such problem: establishing dense visual correspondence across different object instances. For this task, although we do not know what the ground-truth is, we know it should be consistent across instances of that category. We exploit this consistency as a supervisory signal to train a convolutional neural network to predict cross-instance correspondences between pairs of images depicting objects of the same category. For each pair of training images we find an appropriate 3D CAD model and render two synthetic views to link in with the pair, establishing a correspondence flow 4-cycle. We use ground-truth synthetic-to-synthetic correspondences, provided by the rendering engine, to train a ConvNet to predict synthetic-to-real, real-to-real and real-to-synthetic correspondences that are cycle-consistent with the ground-truth. At test time, no CAD models are required. We demonstrate that our end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods in correspondence-related tasks.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

research
04/07/2021

Warp Consistency for Unsupervised Learning of Dense Correspondences

The key challenge in learning dense correspondences lies in the lack of ...
research
01/07/2019

All Graphs Lead to Rome: Learning Geometric and Cycle-Consistent Representations with Graph Convolutional Networks

Image feature matching is a fundamental part of many geometric computer ...
research
05/19/2017

Matching neural paths: transfer from recognition to correspondence search

Many machine learning tasks require finding per-part correspondences bet...
research
12/14/2020

Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings

Cycle-consistent training is widely used for jointly learning a forward ...
research
06/17/2021

Discovering Relationships between Object Categories via Universal Canonical Maps

We tackle the problem of learning the geometry of multiple categories of...
research
07/23/2019

Canonical Surface Mapping via Geometric Cycle Consistency

We explore the task of Canonical Surface Mapping (CSM). Specifically, gi...
research
02/06/2014

Near-Optimal Joint Object Matching via Convex Relaxation

Joint matching over a collection of objects aims at aggregating informat...

Please sign up or login with your details

Forgot password? Click here to reset