Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence

10/06/2022
by   Sunghwan Hong, et al.
0

Existing pipelines of semantic correspondence commonly include extracting high-level semantic features for the invariance against intra-class variations and background clutters. This architecture, however, inevitably results in a low-resolution matching field that additionally requires an ad-hoc interpolation process as a post-processing for converting it into a high-resolution one, certainly limiting the overall performance of matching results. To overcome this, inspired by recent success of implicit neural representation, we present a novel method for semantic correspondence, called Neural Matching Field (NeMF). However, complicacy and high-dimensionality of a 4D matching field are the major hindrances, which we propose a cost embedding network to process a coarse cost volume to use as a guidance for establishing high-precision matching field through the following fully-connected network. Nevertheless, learning a high-dimensional matching field remains challenging mainly due to computational complexity, since a naive exhaustive inference would require querying from all pixels in the 4D space to infer pixel-wise correspondences. To overcome this, we propose adequate training and inference procedures, which in the training phase, we randomly sample matching candidates and in the inference phase, we iteratively performs PatchMatch-based inference and coordinate optimization at test time. With these combined, competitive results are attained on several standard benchmarks for semantic correspondence. Code and pre-trained weights are available at https://ku-cvlab.github.io/NeMF/.

READ FULL TEXT

page 3

page 7

page 9

page 18

page 19

page 20

research
09/11/2021

Convolutional Hough Matching Networks for Robust and Efficient Visual Correspondence

Despite advances in feature representation, leveraging geometric relatio...
research
05/10/2023

View Correspondence Network for Implicit Light Field Representation

We present a novel technique for implicit neural representation of light...
research
03/31/2021

Convolutional Hough Matching Networks

Despite advances in feature representation, leveraging geometric relatio...
research
06/06/2021

Deep Matching Prior: Test-Time Optimization for Dense Correspondence

Conventional techniques to establish dense correspondences across visual...
research
05/19/2017

Matching neural paths: transfer from recognition to correspondence search

Many machine learning tasks require finding per-part correspondences bet...
research
07/21/2020

Learning to Compose Hypercolumns for Visual Correspondence

Feature representation plays a crucial role in visual correspondence, an...
research
12/17/2020

𝕏Resolution Correspondence Networks

In this paper, we aim at establishing accurate dense correspondences bet...

Please sign up or login with your details

Forgot password? Click here to reset