Digging Into Self-Supervised Learning of Feature Descriptors

10/10/2021
by   Iaroslav Melekhov, et al.
0

Fully-supervised CNN-based approaches for learning local image descriptors have shown remarkable results in a wide range of geometric tasks. However, most of them require per-pixel ground-truth keypoint correspondence data which is difficult to acquire at scale. To address this challenge, recent weakly- and self-supervised methods can learn feature descriptors from relative camera poses or using only synthetic rigid transformations such as homographies. In this work, we focus on understanding the limitations of existing self-supervised approaches and propose a set of improvements that combined lead to powerful feature descriptors. We show that increasing the search space from in-pair to in-batch for hard negative mining brings consistent improvement. To enhance the discriminativeness of feature descriptors, we propose a coarse-to-fine method for mining local hard negatives from a wider search space by using global visual image descriptors. We demonstrate that a combination of synthetic homography transformation, color augmentation, and photorealistic image stylization produces useful representations that are viewpoint and illumination invariant. The feature descriptors learned by the proposed approach perform competitively and surpass their fully- and weakly-supervised counterparts on various geometric benchmarks such as image-based localization, sparse feature matching, and image retrieval.

READ FULL TEXT

page 2

page 6

page 11

page 12

page 13

page 14

page 15

research
04/28/2020

Learning Feature Descriptors using Camera Pose Supervision

Recent research on learned visual descriptors has shown promising improv...
research
09/12/2022

Learning Dense Visual Descriptors using Image Augmentations for Robot Manipulation Tasks

We propose a self-supervised training approach for learning view-invaria...
research
02/03/2017

FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence

We present a descriptor, called fully convolutional self-similarity (FCS...
research
01/25/2022

Self-supervised Point Cloud Registration with Deep Versatile Descriptors

Recent years have witnessed an increasing trend toward solving point clo...
research
12/08/2022

DDM-NET: End-to-end learning of keypoint feature Detection, Description and Matching for 3D localization

In this paper, we propose an end-to-end framework that jointly learns ke...
research
08/04/2018

Learning to Align Images using Weak Geometric Supervision

Image alignment tasks require accurate pixel correspondences, which are ...
research
04/07/2021

LIFE: Lighting Invariant Flow Estimation

We tackle the problem of estimating flow between two images with large l...

Please sign up or login with your details

Forgot password? Click here to reset