DeepAI AI Chat
Log In Sign Up

Hierarchical Metric Learning and Matching for 2D and 3D Geometric Correspondences

by   Mohammed E. Fathy, et al.

Interest point descriptors have fueled progress on almost every problem in computer vision. Recent advances in deep neural networks have enabled task-specific learned descriptors that outperform hand-crafted descriptors on many problems. We demonstrate that commonly used metric learning approaches do not optimally leverage the feature hierarchies learned in a Convolutional Neural Network (CNN), especially when applied to the task of geometric feature matching. While a metric loss applied to the deepest layer of a CNN, is often expected to yield ideal features irrespective of the task, in fact the growing receptive field as well as striding effects cause shallower features to be better at high precision matching tasks. We leverage this insight together with explicit supervision at multiple levels of the feature hierarchy for better regularization, to learn more effective descriptors in the context of geometric matching tasks. Further, we propose to use activation maps at different layers of a CNN, as an effective and principled replacement for the multi-resolution image pyramids often used for matching tasks. We propose concrete CNN architectures employing these ideas, and evaluate them on multiple datasets for 2D and 3D geometric matching as well as optical flow, demonstrating state-of-the-art results and generalization across datasets.


page 2

page 5

page 8

page 11

page 12

page 19


Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT

Latest results indicate that features learned via convolutional neural n...

Learning a Local Feature Descriptor for 3D LiDAR Scans

Robust data association is necessary for virtually every SLAM system and...

Learning an Effective Equivariant 3D Descriptor Without Supervision

Establishing correspondences between 3D shapes is a fundamental task in ...

SDC - Stacked Dilated Convolution: A Unified Descriptor Network for Dense Matching Tasks

Dense pixel matching is important for many computer vision tasks such as...

DeepPoint3D: Learning Discriminative Local Descriptors using Deep Metric Learning on 3D Point Clouds

Learning local descriptors is an important problem in computer vision. W...

Image Patch Matching Using Convolutional Descriptors with Euclidean Distance

In this work we propose a neural network based image descriptor suitable...

Scalable Full Flow with Learned Binary Descriptors

We propose a method for large displacement optical flow in which local m...