Investigating the Role of Image Retrieval for Visual Localization – An exhaustive benchmark

05/31/2022
by   Martin Humenberger, et al.
0

Visual localization, i.e., camera pose estimation in a known scene, is a core component of technologies such as autonomous driving and augmented reality. State-of-the-art localization approaches often rely on image retrieval techniques for one of two purposes: (1) provide an approximate pose estimate or (2) determine which parts of the scene are potentially visible in a given query image. It is common practice to use state-of-the-art image retrieval algorithms for both of them. These algorithms are often trained for the goal of retrieving the same landmark under a large range of viewpoint changes which often differs from the requirements of visual localization. In order to investigate the consequences for visual localization, this paper focuses on understanding the role of image retrieval for multiple visual localization paradigms. First, we introduce a novel benchmark setup and compare state-of-the-art retrieval representations on multiple datasets using localization performance as metric. Second, we investigate several definitions of "ground truth" for image retrieval. Using these definitions as upper bounds for the visual localization paradigms, we show that there is still sgnificant room for improvement. Third, using these tools and in-depth analysis, we show that retrieval performance on classical landmark retrieval or place recognition tasks correlates only for some but not all paradigms to localization performance. Finally, we analyze the effects of blur and dynamic scenes in the images. We conclude that there is a need for retrieval approaches specifically designed for localization paradigms. Our benchmark and evaluation protocols are available at https://github.com/naver/kapture-localization.

READ FULL TEXT

page 2

page 21

page 25

page 26

research
11/24/2020

Benchmarking Image Retrieval for Visual Localization

Visual localization, i.e., camera pose estimation in a known scene, is a...
research
07/27/2020

Robust Image Retrieval-based Visual Localization using Kapture

In this paper, we present a versatile method for visual localization. It...
research
02/25/2021

Scene Retrieval for Contextual Visual Mapping

Visual navigation localizes a query place image against a reference data...
research
04/12/2023

Visual Localization using Imperfect 3D Models from the Internet

Visual localization is a core component in many applications, including ...
research
04/13/2022

Reuse your features: unifying retrieval and feature-metric alignment

We propose a compact pipeline to unify all the steps of Visual Localizat...
research
07/28/2017

Benchmarking 6DOF Urban Visual Localization in Changing Conditions

Visual localization enables autonomous vehicles to navigate in their sur...
research
08/15/2016

Detecting Dominant Vanishing Points in Natural Scenes with Application to Composition-Sensitive Image Retrieval

Linear perspective is widely used in landscape photography to create the...

Please sign up or login with your details

Forgot password? Click here to reset