Log In Sign Up

COTR: Correspondence Transformer for Matching Across Images

by   Wei Jiang, et al.

We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence in the other. By doing so, one has the option to query only the points of interest and retrieve sparse correspondences, or to query all points in an image and obtain dense mappings. Importantly, in order to capture both local and global priors, and to let our model relate between image regions using the most relevant among said priors, we realize our network using a transformer. At inference time, we apply our correspondence network by recursively zooming in around the estimates, yielding a multiscale pipeline able to provide highly-accurate correspondences. Our method significantly outperforms the state of the art on both sparse and dense correspondence problems on multiple datasets and tasks, ranging from wide-baseline stereo to optical flow, without any retraining for a specific dataset. We commit to releasing data, code, and all the tools necessary to train from scratch and ensure reproducibility.


page 1

page 4

page 6

page 7

page 8


ECO-TR: Efficient Correspondences Finding Via Coarse-to-Fine Refinement

Modeling sparse and dense image matching within a unified functional cor...

GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network

The feature correlation layer serves as a key neural network module in n...

NeuralMarker: A Framework for Learning General Marker Correspondence

We tackle the problem of estimating correspondences from a general marke...

LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation

Cross-resolution image alignment is a key problem in multiscale gigapixe...

Matching neural paths: transfer from recognition to correspondence search

Many machine learning tasks require finding per-part correspondences bet...

Consensus-Guided Correspondence Denoising

Correspondence selection between two groups of feature points aims to co...

Wide baseline stereo matching with convex bounded-distortion constraints

Finding correspondences in wide baseline setups is a challenging problem...