3DG-STFM: 3D Geometric Guided Student-Teacher Feature Matching

07/06/2022
by   Runyu Mao, et al.
0

We tackle the essential task of finding dense visual correspondences between a pair of images. This is a challenging problem due to various factors such as poor texture, repetitive patterns, illumination variation, and motion blur in practical scenarios. In contrast to methods that use dense correspondence ground-truths as direct supervision for local feature matching training, we train 3DG-STFM: a multi-modal matching model (Teacher) to enforce the depth consistency under 3D dense correspondence supervision and transfer the knowledge to 2D unimodal matching model (Student). Both teacher and student models consist of two transformer-based matching modules that obtain dense correspondences in a coarse-to-fine manner. The teacher model guides the student model to learn RGB-induced depth information for the matching purpose on both coarse and fine branches. We also evaluate 3DG-STFM on a model compression task. To the best of our knowledge, 3DG-STFM is the first student-teacher learning method for the local feature matching task. The experiments show that our method outperforms state-of-the-art methods on indoor and outdoor camera pose estimations, and homography estimation problems. Code is available at: https://github.com/Ryan-prime/3DG-STFM.

READ FULL TEXT
research
05/23/2019

Zero-shot Knowledge Transfer via Adversarial Belief Matching

Performing knowledge transfer from a large teacher network to a smaller ...
research
04/01/2021

LoFTR: Detector-Free Local Feature Matching with Transformers

We present a novel method for local image feature matching. Instead of p...
research
07/01/2022

TopicFM: Robust and Interpretable Feature Matching with Topic-assisted

Finding correspondences across images is an important task in many visua...
research
09/25/2022

ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement

Modeling sparse and dense image matching within a unified functional cor...
research
02/01/2022

Local Feature Matching with Transformers for low-end devices

LoFTR arXiv:2104.00680 is an efficient deep learning method for finding ...
research
12/17/2020

𝕏Resolution Correspondence Networks

In this paper, we aim at establishing accurate dense correspondences bet...
research
05/24/2023

RoMa: Revisiting Robust Losses for Dense Feature Matching

Dense feature matching is an important computer vision task that involve...

Please sign up or login with your details

Forgot password? Click here to reset