APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulation

05/31/2021
by   Yizhe Wu, et al.
1

Recent advances in unsupervised learning for object detection, segmentation, and tracking hold significant promise for applications in robotics. A common approach is to frame these tasks as inference in probabilistic latent-variable models. In this paper, however, we show that the current state-of-the-art struggles with visually complex scenes such as typically encountered in robot manipulation tasks. We propose APEX, a new latent-variable model which is able to segment and track objects in more realistic scenes featuring objects that vary widely in size and texture, including the robot arm itself. This is achieved by a principled mask normalisation algorithm and a high-resolution scene encoder. To evaluate our approach, we present results on the real-world Sketchy dataset. This dataset, however, does not contain ground truth masks and object IDs for a quantitative evaluation. We thus introduce the Panda Pushing Dataset (P2D) which shows a Panda arm interacting with objects on a table in simulation and which includes ground-truth segmentation masks and object IDs for tracking. In both cases, APEX comprehensively outperforms the current state-of-the-art in unsupervised object segmentation and tracking. We demonstrate the efficacy of our segmentations for robot skill execution on an object arrangement task, where we also achieve the best or comparable performance among all the baselines.

READ FULL TEXT

page 1

page 2

page 4

page 5

page 7

research
06/07/2022

ObPose: Leveraging Canonical Pose for Object-Centric Scene Inference in 3D

We present ObPose, an unsupervised object-centric generative model that ...
research
06/12/2020

Unmasking the Inductive Biases of Unsupervised Object Representations for Video Sequences

Perceiving the world in terms of objects is a crucial prerequisite for r...
research
11/28/2021

Learning To Segment Dominant Object Motion From Watching Videos

Existing deep learning based unsupervised video object segmentation meth...
research
11/19/2021

ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation

There has been a recent surge in methods that aim to decompose and segme...
research
04/01/2021

Fusing RGBD Tracking and Segmentation Tree Sampling for Multi-Hypothesis Volumetric Segmentation

Despite rapid progress in scene segmentation in recent years, 3D segment...
research
08/21/2023

Unsupervised Dialogue Topic Segmentation in Hyperdimensional Space

We present HyperSeg, a hyperdimensional computing (HDC) approach to unsu...
research
03/28/2020

Refined Plane Segmentation for Cuboid-Shaped Objects by Leveraging Edge Detection

Recent advances in the area of plane segmentation from single RGB images...

Please sign up or login with your details

Forgot password? Click here to reset