Log In Sign Up

Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty

by   Bingbing Zhuang, et al.

Learning methods for relative camera pose estimation have been developed largely in isolation from classical geometric approaches. The question of how to integrate predictions from deep neural networks (DNNs) and solutions from geometric solvers, such as the 5-point algorithm, has as yet remained under-explored. In this paper, we present a novel framework that involves probabilistic fusion between the two families of predictions during network training, with a view to leveraging their complementary benefits in a learnable way. The fusion is achieved by learning the DNN uncertainty under explicit guidance by the geometric uncertainty, thereby learning to take into account the geometric solution in relation to the DNN prediction. Our network features a self-attention graph neural network, which drives the learning by enforcing strong interactions between different correspondences and potentially modeling complex relationships between points. We propose motion parmeterizations suitable for learning and show that our method achieves state-of-the-art performance on the challenging DeMoN and ScanNet datasets. While we focus on relative pose, we envision that our pipeline is broadly applicable for fusing classical geometry and deep learning.


page 4

page 8


DeepPose: Human Pose Estimation via Deep Neural Networks

We propose a method for human pose estimation based on Deep Neural Netwo...

Leveraging Equivariant Features for Absolute Pose Regression

While end-to-end approaches have achieved state-of-the-art performance i...

Deep Two-View Structure-from-Motion Revisited

Two-view structure-from-motion (SfM) is the cornerstone of 3D reconstruc...

Fusing Convolutional Neural Network and Geometric Constraint for Image-based Indoor Localization

This paper proposes a new image-based localization framework that explic...

One Versus all for deep Neural Network Incertitude (OVNNI) quantification

Deep neural networks (DNNs) are powerful learning models yet their resul...

Predicting Out-of-View Feature Points for Model-Based Camera Pose Estimation

In this work we present a novel framework that uses deep learning to pre...