Log In Sign Up

GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network

by   Prune Truong, et al.

The feature correlation layer serves as a key neural network module in numerous computer vision problems that involve dense correspondences between image pairs. It predicts a correspondence volume by evaluating dense scalar products between feature vectors extracted from pairs of locations in two images. However, this point-to-point feature comparison is insufficient when disambiguating multiple similar regions in an image, severely affecting the performance of the end task. We propose GOCor, a fully differentiable dense matching module, acting as a direct replacement to the feature correlation layer. The correspondence volume generated by our module is the result of an internal optimization procedure that explicitly accounts for similar regions in the scene. Moreover, our approach is capable of effectively learning spatial matching priors to resolve further matching ambiguities. We analyze our GOCor module in extensive ablative experiments. When integrated into state-of-the-art networks, our approach significantly outperforms the feature correlation layer for the tasks of geometric matching, optical flow, and dense semantic matching. The code and trained models will be made available at


page 2

page 9

page 26

page 30

page 31

page 32

page 33

page 34


COTR: Correspondence Transformer for Matching Across Images

We propose a novel framework for finding correspondences in images based...

Learning Optical Flow from a Few Matches

State-of-the-art neural network models for optical flow estimation requi...

Adaptive Assignment for Geometry Aware Local Feature Matching

Local image feature matching, aiming to identify and correspond similar ...

GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences

Establishing dense correspondences between a pair of images is an import...

Warp Consistency for Unsupervised Learning of Dense Correspondences

The key challenge in learning dense correspondences lies in the lack of ...

SeFM: A Sequential Feature Point Matching Algorithm for Object 3D Reconstruction

3D reconstruction is a fundamental issue in many applications and the fe...

Linking Image and Text with 2-Way Nets

Linking two data sources is a basic building block in numerous computer ...