Object-level 3D Semantic Mapping using a Network of Smart Edge Sensors

11/21/2022
by   Julian Hau, et al.
0

Autonomous robots that interact with their environment require a detailed semantic scene model. For this, volumetric semantic maps are frequently used. The scene understanding can further be improved by including object-level information in the map. In this work, we extend a multi-view 3D semantic mapping system consisting of a network of distributed smart edge sensors with object-level information, to enable downstream tasks that need object-level input. Objects are represented in the map via their 3D mesh model or as an object-centric volumetric sub-map that can model arbitrary object geometry when no detailed 3D model is available. We propose a keypoint-based approach to estimate object poses via PnP and refinement via ICP alignment of the 3D object model with the observed point cloud segments. Object instances are tracked to integrate observations over time and to be robust against temporary occlusions. Our method is evaluated on the public Behave dataset where it shows pose estimation accuracy within a few centimeters and in real-world experiments with the sensor network in a challenging lab environment where multiple chairs and a table are tracked through the scene online, in real time even under high occlusions.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

page 8

research
05/03/2022

3D Semantic Scene Perception using Distributed Smart Edge Sensors

We present a system for 3D semantic scene perception consisting of a net...
research
03/01/2019

Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery

To autonomously navigate and plan interactions in real-world environment...
research
09/21/2021

Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency

For robotic interaction in an environment shared with multiple agents, a...
research
10/26/2018

Semantic Mapping with Simultaneous Object Detection and Localization

We present a filtering-based method for semantic mapping to simultaneous...
research
05/02/2022

POCD: Probabilistic Object-Level Change Detection and Volumetric Mapping in Semi-Static Scenes

Maintaining an up-to-date map to reflect recent changes in the scene is ...
research
09/14/2023

Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions

Perceiving and manipulating 3D articulated objects in diverse environmen...
research
11/18/2014

Towards Scene Understanding with Detailed 3D Object Representations

Current approaches to semantic image and scene understanding typically e...

Please sign up or login with your details

Forgot password? Click here to reset