Understanding the Limitations of CNN-based Absolute Camera Pose Regression

03/18/2019
by   Torsten Sattler, et al.
26

Visual localization is the task of accurate camera pose estimation in a known scene. It is a key problem in computer vision and robotics, with applications including self-driving cars, Structure-from-Motion, SLAM, and Mixed Reality. Traditionally, the localization problem has been tackled using 3D geometry. Recently, end-to-end approaches based on convolutional neural networks have become popular. These methods learn to directly regress the camera pose from an input image. However, they do not achieve the same level of pose accuracy as 3D structure-based methods. To understand this behavior, we develop a theoretical model for camera pose regression. We use our model to predict failure cases for pose regression techniques and verify our predictions through experiments. We furthermore use our model to show that pose regression is more closely related to pose approximation via image retrieval than to accurate pose estimation via 3D structure. A key result is that current approaches do not consistently outperform a handcrafted image retrieval baseline. This clearly shows that additional research is needed before pose regression algorithms are ready to compete with structure-based methods.

READ FULL TEXT

page 3

page 6

research
01/15/2022

A Critical Analysis of Image-based Camera Pose Estimation Techniques

Camera, and associated with its objects within the field of view, locali...
research
04/05/2022

Leveraging Equivariant Features for Absolute Pose Regression

While end-to-end approaches have achieved state-of-the-art performance i...
research
08/04/2019

To Learn or Not to Learn: Visual Localization from Essential Matrices

Visual localization is the problem of estimating a camera within a scene...
research
03/19/2021

CoordiNet: uncertainty-aware pose regressor for reliable vehicle localization

In this paper, we investigate visual-based camera localization with neur...
research
09/23/2019

How to improve CNN-based 6-DoF camera pose estimation

Convolutional neural networks (CNNs) and transfer learning have recently...
research
12/22/2020

A Structure-Aware Method for Direct Pose Estimation

Estimating camera pose from a single image is a fundamental problem in c...
research
07/01/2021

Deep auxiliary learning for visual localization using colorization task

Visual localization is one of the most important components for robotics...

Please sign up or login with your details

Forgot password? Click here to reset