SEKD: Self-Evolving Keypoint Detection and Description

by   Yafei Song, et al.
Alibaba Group
Peking University

Researchers have attempted utilizing deep neural network (DNN) to learn novel local features from images inspired by its recent successes on a variety of vision tasks. However, existing DNN-based algorithms have not achieved such remarkable progress that could be partly attributed to insufficient utilization of the interactive characters between local feature detector and descriptor. To alleviate these difficulties, we emphasize two desired properties, i.e., repeatability and reliability, to simultaneously summarize the inherent and interactive characters of local feature detector and descriptor. Guided by these properties, a self-supervised framework, namely self-evolving keypoint detection and description (SEKD), is proposed to learn an advanced local feature model from unlabeled natural images. Additionally, to have performance guarantees, novel training strategies have also been dedicatedly designed to minimize the gap between the learned feature and its properties. We benchmark the proposed method on homography estimation, relative pose estimation, and structure-from-motion tasks. Extensive experimental results demonstrate that the proposed method outperforms popular hand-crafted and DNN-based methods by remarkable margins. Ablation studies also verify the effectiveness of each critical training strategy. We will release our code along with the trained model publicly.


page 1

page 4

page 5

page 6


Neural Outlier Rejection for Self-Supervised Keypoint Learning

Identifying salient points in images is a crucial component for visual o...

FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images

Keypoint detection and description is a commonly used building block in ...

S-TREK: Sequential Translation and Rotation Equivariant Keypoints for local feature extraction

In this work we introduce S-TREK, a novel local feature extractor that c...

On the Comparison of Classic and Deep Keypoint Detector and Descriptor Methods

The purpose of this study is to give a performance comparison between se...

HandMIM: Pose-Aware Self-Supervised Learning for 3D Hand Mesh Estimation

With an enormous number of hand images generated over time, unleashing p...

Robust Keypoint Detection and Pose Estimation of Robot Manipulators with Self-Occlusions via Sim-to-Real Transfer

Keypoint detection is an essential building block for many robotic appli...

Please sign up or login with your details

Forgot password? Click here to reset