Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localization

05/19/2020
by   Peter Karkus, et al.
0

Mapping and localization, preferably from a small number of observations, are fundamental tasks in robotics. We address these tasks by combining spatial structure (differentiable mapping) and end-to-end learning in a novel neural network architecture: the Differentiable Mapping Network (DMN). The DMN constructs a spatially structured view-embedding map and uses it for subsequent visual localization with a particle filter. Since the DMN architecture is end-to-end differentiable, we can jointly learn the map representation and localization using gradient descent. We apply the DMN to sparse visual localization, where a robot needs to localize in a new environment with respect to a small number of images from known viewpoints. We evaluate the DMN using simulated environments and a challenging real-world Street View dataset. We find that the DMN learns effective map representations for visual localization. The benefit of spatial structure increases with larger environments, more viewpoints for mapping, and when training data is scarce. Project website: http://sites.google.com/view/differentiable-mapping

READ FULL TEXT

page 1

page 2

page 3

research
04/24/2020

Towards Differentiable Resampling

Resampling is a key component of sample-based recursive state estimation...
research
05/28/2019

Differentiable Algorithm Networks for Composable Robot Learning

This paper introduces the Differentiable Algorithm Network (DAN), a comp...
research
05/23/2018

Particle Filter Networks with Application to Visual Localization

Particle filtering is a powerful method for sequential state estimation ...
research
01/24/2018

Active Neural Localization

Localization is the problem of estimating the location of an autonomous ...
research
05/23/2018

Particle Filter Networks: End-to-End Probabilistic Localization From Visual Observations

Particle filters sequentially approximate posterior distributions by sam...
research
12/02/2018

ECO: Egocentric Cognitive Mapping

We present a new method to localize a camera within a previously unseen ...
research
07/19/2022

Theseus: A Library for Differentiable Nonlinear Optimization

We present Theseus, an efficient application-agnostic open source librar...

Please sign up or login with your details

Forgot password? Click here to reset