Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs

01/18/2021
by   Antoni Rosinol, et al.
28

Humans are able to form a complex mental model of the environment they move in. This mental model captures geometric and semantic aspects of the scene, describes the environment at multiple levels of abstractions (e.g., objects, rooms, buildings), includes static and dynamic entities and their relations (e.g., a person is in a room at a given time). In contrast, current robots' internal representations still provide a partial and fragmented understanding of the environment, either in the form of a sparse or dense set of geometric primitives (e.g., points, lines, planes, voxels) or as a collection of objects. This paper attempts to reduce the gap between robot and human perception by introducing a novel representation, a 3D Dynamic Scene Graph(DSG), that seamlessly captures metric and semantic aspects of a dynamic environment. A DSG is a layered graph where nodes represent spatial concepts at different levels of abstraction, and edges represent spatio-temporal relations among nodes. Our second contribution is Kimera, the first fully automatic method to build a DSG from visual-inertial data. Kimera includes state-of-the-art techniques for visual-inertial SLAM, metric-semantic 3D reconstruction, object localization, human pose and shape estimation, and scene parsing. Our third contribution is a comprehensive evaluation of Kimera in real-life datasets and photo-realistic simulations, including a newly released dataset, uHumans2, which simulates a collection of crowded indoor and outdoor scenes. Our evaluation shows that Kimera achieves state-of-the-art performance in visual-inertial SLAM, estimates an accurate 3D metric-semantic mesh model in real-time, and builds a DSG of a complex indoor environment with tens of objects and humans in minutes. Our final contribution shows how to use a DSG for real-time hierarchical semantic path-planning. The core modules in Kimera are open-source.

READ FULL TEXT

page 4

page 6

page 14

page 16

page 23

page 24

page 25

page 26

research
02/15/2020

3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans

We present a unified representation for actionable spatial perception: 3...
research
10/06/2019

Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping

We provide an open-source C++ library for real-time metric-semantic visu...
research
05/11/2023

Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systems

3D spatial perception is the problem of building and maintaining an acti...
research
01/31/2022

Hydra: A Real-time Spatial Perception Engine for 3D Scene Graph Construction and Optimization

3D scene graphs have recently emerged as a powerful high-level represent...
research
06/28/2021

Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems

This paper presents Kimera-Multi, the first multi-robot system that (i) ...
research
04/26/2023

Hydra-Multi: Collaborative Online Construction of 3D Scene Graphs with Multi-Robot Teams

3D scene graphs have recently emerged as an expressive high-level map re...
research
11/16/2022

Advanced Situational Graphs for Robot Navigation in Structured Indoor Environments

Mobile robots extract information from its environment to understand the...

Please sign up or login with your details

Forgot password? Click here to reset