PDC-Net+: Enhanced Probabilistic Dense Correspondence Network

09/28/2021
by   Prune Truong, et al.
14

Establishing robust and accurate correspondences between a pair of images is a long-standing computer vision problem with numerous applications. While classically dominated by sparse methods, emerging dense approaches offer a compelling alternative paradigm that avoids the keypoint detection step. However, dense flow estimation is often inaccurate in the case of large displacements, occlusions, or homogeneous regions. In order to apply dense methods to real-world applications, such as pose estimation, image manipulation, or 3D reconstruction, it is therefore crucial to estimate the confidence of the predicted matches. We propose the Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, capable of estimating accurate dense correspondences along with a reliable confidence map. We develop a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty. In particular, we parametrize the predictive distribution as a constrained mixture model, ensuring better modelling of both accurate flow predictions and outliers. Moreover, we develop an architecture and an enhanced training strategy tailored for robust and generalizable uncertainty prediction in the context of self-supervised training. Our approach obtains state-of-the-art results on multiple challenging geometric matching and optical flow datasets. We further validate the usefulness of our probabilistic confidence estimation for the tasks of pose estimation, 3D reconstruction, image-based localization, and image retrieval. Code and models are available at https://github.com/PruneTruong/DenseMatching.

READ FULL TEXT

page 28

page 29

page 30

page 31

page 32

page 33

page 34

page 35

research
01/05/2021

Learning Accurate Dense Correspondences and When to Trust Them

Establishing dense correspondences between a pair of images is an import...
research
06/07/2023

Learning Probabilistic Coordinate Fields for Robust Correspondences

We introduce Probabilistic Coordinate Fields (PCFs), a novel geometric-i...
research
12/09/2021

ScaleNet: A Shallow Architecture for Scale Estimation

In this paper, we address the problem of estimating scale factors betwee...
research
10/19/2018

DGC-Net: Dense Geometric Correspondence Network

This paper addresses the challenge of dense pixel correspondence estimat...
research
11/01/2020

Robust localization for planar moving robot in changing environment: A perspective on density of correspondence and depth

Visual localization for planar moving robot is important to various indo...
research
08/25/2020

Confidence-aware Adversarial Learning for Self-supervised Semantic Matching

In this paper, we aim to address the challenging task of semantic matchi...
research
06/08/2022

PixSelect: Less but Reliable Pixels for Accurate and Efficient Localization

Accurate camera pose estimation is a fundamental requirement for numerou...

Please sign up or login with your details

Forgot password? Click here to reset