Log In Sign Up

Boosting LiDAR-based Semantic Labeling by Cross-Modal Training Data Generation

by   Florian Piewak, et al.

Mobile robots and autonomous vehicles rely on multi-modal sensor setups to perceive and understand their surroundings. Aside from cameras, LiDAR sensors represent a central component of state-of-the-art perception systems. In addition to accurate spatial perception, a comprehensive semantic understanding of the environment is essential for efficient and safe operation. In this paper we present a novel deep neural network architecture called LiLaNet for point-wise, multi-class semantic labeling of semi-dense LiDAR data. The network utilizes virtual image projections of the 3D point clouds for efficient inference. Further, we propose an automated process for large-scale cross-modal training data generation called Autolabeling, in order to boost semantic labeling performance while keeping the manual annotation effort low. The effectiveness of the proposed network architecture as well as the automated data generation process is demonstrated on a manually annotated ground truth dataset. LiLaNet is shown to significantly outperform current state-of-the-art CNN architectures for LiDAR data. Applying our automatically generated large-scale training data yields a boost of up to 14 percentage points compared to networks trained on manually annotated data only.


page 6

page 8

page 9

page 12


Analyzing the Cross-Sensor Portability of Neural Network Architectures for LiDAR-based Semantic Labeling

State-of-the-art approaches for the semantic labeling of LiDAR point clo...

EfficientLPS: Efficient LiDAR Panoptic Segmentation

Panoptic segmentation of point clouds is a crucial task that enables aut...

Automatic Labeled LiDAR Data Generation based on Precise Human Model

Following improvements in deep neural networks, state-of-the-art network...

Large-Scale 3D Semantic Reconstruction for Automated Driving Vehicles with Adaptive Truncated Signed Distance Function

The Large-scale 3D reconstruction, texturing and semantic mapping are no...

Contrastive Learning of Features between Images and LiDAR

Image and Point Clouds provide different information for robots. Finding...

Drive Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillation

This work investigates learning pixel-wise semantic image segmentation i...

Multi-modal Geolocation Estimation Using Deep Neural Networks

Estimating the location where an image was taken based solely on the con...