LabelFusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes

by   Pat Marion, et al.

Deep neural network (DNN) architectures have been shown to outperform traditional pipelines for object segmentation and pose estimation using RGBD data, but the performance of these DNN pipelines is directly tied to how representative the training data is of the true data. Hence a key requirement for employing these methods in practice is to have a large set of labeled data for your specific robotic manipulation task, a requirement that is not generally satisfied by existing datasets. In this paper we develop a pipeline to rapidly generate high quality RGBD data with pixelwise labels and object poses. We use an RGBD camera to collect video of a scene from multiple viewpoints and leverage existing reconstruction techniques to produce a 3D dense reconstruction. We label the 3D reconstruction using a human assisted ICP-fitting of object meshes. By reprojecting the results of labeling the 3D scene we can produce labels for each RGBD image of the scene. This pipeline enabled us to collect over 1,000,000 labeled object instances in just a few days. We use this dataset to answer questions related to how much training data is required, and of what quality the data must be, to achieve high performance from a DNN architecture.


page 3

page 5

page 6

page 7


Stillleben: Realistic Scene Synthesis for Deep Learning in Robotics

Training data is the key ingredient for deep learning approaches, but di...

Rapid Pose Label Generation through Sparse Representation of Unknown Objects

Deep Convolutional Neural Networks (CNNs) have been successfully deploye...

Physics-based Scene-level Reasoning for Object Pose Estimation in Clutter

This paper focuses on vision-based pose estimation for multiple rigid ob...

Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation

Understanding the scene is key for autonomously navigating vehicles and ...

Using ScrutinAI for Visual Inspection of DNN Performance in a Medical Use Case

Our Visual Analytics (VA) tool ScrutinAI supports human analysts to inve...

Automatic Labeled LiDAR Data Generation based on Precise Human Model

Following improvements in deep neural networks, state-of-the-art network...

DeepLight: Robust Unobtrusive Real-time Screen-Camera Communication for Real-World Displays

The paper introduces a novel, holistic approach for robust Screen-Camera...

Please sign up or login with your details

Forgot password? Click here to reset