DeepAI
Log In Sign Up

COTR: Correspondence Transformer for Matching Across Images

03/25/2021
by   Wei Jiang, et al.
0

We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence in the other. By doing so, one has the option to query only the points of interest and retrieve sparse correspondences, or to query all points in an image and obtain dense mappings. Importantly, in order to capture both local and global priors, and to let our model relate between image regions using the most relevant among said priors, we realize our network using a transformer. At inference time, we apply our correspondence network by recursively zooming in around the estimates, yielding a multiscale pipeline able to provide highly-accurate correspondences. Our method significantly outperforms the state of the art on both sparse and dense correspondence problems on multiple datasets and tasks, ranging from wide-baseline stereo to optical flow, without any retraining for a specific dataset. We commit to releasing data, code, and all the tools necessary to train from scratch and ensure reproducibility.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

09/25/2022

ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement

Modeling sparse and dense image matching within a unified functional cor...
09/16/2020

GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network

The feature correlation layer serves as a key neural network module in n...
09/19/2022

NeuralMarker: A Framework for Learning General Marker Correspondence

We tackle the problem of estimating correspondences from a general marke...
06/08/2021

LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation

Cross-resolution image alignment is a key problem in multiscale gigapixe...
05/19/2017

Matching neural paths: transfer from recognition to correspondence search

Many machine learning tasks require finding per-part correspondences bet...
01/03/2021

Consensus-Guided Correspondence Denoising

Correspondence selection between two groups of feature points aims to co...
06/10/2015

Wide baseline stereo matching with convex bounded-distortion constraints

Finding correspondences in wide baseline setups is a challenging problem...