From Pixels to Buildings: End-to-end Probabilistic Deep Networks for Large-scale Semantic Mapping

12/31/2018

∙

by   Kaiyu Zheng, et al.

∙

0

∙

share

We introduce TopoNets, end-to-end probabilistic deep networks for modeling semantic maps with structure reflecting the topology of large-scale environments. TopoNets build unified deep networks spanning multiple levels of abstraction and spatial scales, from pixels representing geometry of local places to high-level descriptions representing semantics of buildings. To this end, TopoNets leverage complex spatial relations expressed in terms of arbitrary, dynamic graphs. We demonstrate how TopoNets can be used to perform end-to-end semantic mapping from partial sensory observations and noisy topological relations discovered by a robot exploring large-scale office spaces. We further illustrate the benefits of the probabilistic representation by generating semantic descriptions augmented with valuable uncertainty information and utilizing likelihoods of complete semantic maps to detect novel and incongruent environment configurations.