Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects

06/20/2017
by   Martin Rünz, et al.
0

In this paper we introduce Co-Fusion, a dense SLAM system that takes a live stream of RGB-D images as input and segments the scene into different objects (using either motion or semantic cues) while simultaneously tracking and reconstructing their 3D shape in real time. We use a multiple model fitting approach where each object can move independently from the background and still be effectively tracked and its shape fused over time using only the information from pixels associated with that object label. Previous attempts to deal with dynamic scenes have typically considered moving regions as outliers, and consequently do not model their shape or track their motion over time. In contrast, we enable the robot to maintain 3D models for each of the segmented objects and to improve them over time through fusion. As a result, our system can enable a robot to maintain a scene description at the object level which has the potential to allow interactions with its working environment; even in the case of dynamic scenes.

READ FULL TEXT

page 1

page 5

page 7

page 8

research
04/24/2018

MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects

We present MaskFusion, a real-time, object-aware, semantic and dynamic R...
research
07/22/2019

DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM

We present DetectFusion, an RGB-D SLAM system that runs in real-time and...
research
10/07/2022

Scene-level Tracking and Reconstruction without Object Priors

We present the first real-time system capable of tracking and reconstruc...
research
08/02/2021

PoseFusion2: Simultaneous Background Reconstruction and Human Shape Recovery in Real-time

Dynamic environments that include unstructured moving objects pose a har...
research
10/06/2022

Feature-Realistic Neural Fusion for Real-Time, Open Set Scene Understanding

General scene understanding for robotics requires flexible semantic repr...
research
10/31/2022

Real-time Mapping of Physical Scene Properties with an Autonomous Robot Experimenter

Neural fields can be trained from scratch to represent the shape and app...
research
04/09/2020

Where Does It End? – Reasoning About Hidden Surfaces by Object Intersection Constraints

Dynamic scene understanding is an essential capability in robotics and V...

Please sign up or login with your details

Forgot password? Click here to reset