DeepAI AI Chat
Log In Sign Up

Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation

04/01/2021
by   Andrew Price, et al.
0

Despite rapid progress in scene segmentation in recent years, 3D segmentation methods are still limited when there is severe occlusion. The key challenge is estimating the segment boundaries of (partially) occluded objects, which are inherently ambiguous when considering only a single frame. In this work, we propose Multihypothesis Segmentation Tracking (MST), a novel method for volumetric segmentation in changing scenes, which allows scene ambiguity to be tracked and our estimates to be adjusted over time as we interact with the scene. Two main innovations allow us to tackle this difficult problem: 1) A novel way to sample possible segmentations from a segmentation tree; and 2) A novel approach to fusing tracking results with multiple segmentation estimates. These methods allow MST to track the segmentation state over time and incorporate new information, such as new objects being revealed. We evaluate our method on several cluttered tabletop environments in simulation and reality. Our results show that MST outperforms baselines in all tested scenes.

READ FULL TEXT

page 3

page 4

page 5

10/21/2020

RigidFusion: Robot Localisation and Mapping in Environments with Large Dynamic Rigid Objects

This work presents a novel approach to simultaneously track a robot with...
05/30/2022

Neural Volumetric Object Selection

We introduce an approach for selecting objects in neural volumetric 3D r...
05/31/2021

APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulation

Recent advances in unsupervised learning for object detection, segmentat...
12/19/2022

Panoptic Lifting for 3D Scene Understanding with Neural Fields

We propose Panoptic Lifting, a novel approach for learning panoptic 3D v...
03/30/2021

SIMstack: A Generative Shape and Instance Model for Unordered Object Stacks

By estimating 3D shape and instances from a single view, we can capture ...
11/01/2020

Tracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints

In order to manipulate a deformable object, such as rope or cloth, in un...
04/09/2018

Binge Watching: Scaling Affordance Learning from Sitcoms

In recent years, there has been a renewed interest in jointly modeling p...