DeepAI AI Chat
Log In Sign Up

R2D2: Repeatable and Reliable Detector and Descriptor

by   Jerome Revaud, et al.

Interest point detection and local feature description are fundamental steps in many computer vision applications. Classical methods for these tasks are based on a detect-then-describe paradigm where separate handcrafted methods are used to first identify repeatable keypoints and then represent them with a local descriptor. Neural networks trained with metric learning losses have recently caught up with these techniques, focusing on learning repeatable saliency maps for keypoint detection and learning descriptors at the detected keypoint locations. In this work, we argue that salient regions are not necessarily discriminative, and therefore can harm the performance of the description. Furthermore, we claim that descriptors should be learned only in regions for which matching can be performed with high confidence. We thus propose to jointly learn keypoint detection and description together with a predictor of the local descriptor discriminativeness. This allows us to avoid ambiguous areas and leads to reliable keypoint detections and descriptions. Our detection-and-description approach, trained with self-supervision, can simultaneously output sparse, repeatable and reliable keypoints that outperforms state-of-the-art detectors and descriptors on the HPatches dataset. It also establishes a record on the recently released Aachen Day-Night localization dataset.


page 2

page 3

page 6

page 7

page 9


UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision

In this paper, we explore how three related tasks, namely keypoint detec...

Realtime CNN-based Keypoint Detector with Sobel Filter and CNN-based Descriptor Trained with Keypoint Candidates

The local feature detector and descriptor are essential in many computer...

FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images

Keypoint detection and description is a commonly used building block in ...

ZippyPoint: Fast Interest Point Detection, Description, and Matching through Mixed Precision Discretization

The design of more complex and powerful neural network models has signif...

End-to-end learning of keypoint detector and descriptor for pose invariant 3D matching

Finding correspondences between images or 3D scans is at the heart of ma...

Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task

We address a core problem of computer vision: Detection and description ...