LiDAR-based Recurrent 3D Semantic Segmentation with Temporal Memory Alignment

03/03/2021
by   Fabian Duerr, et al.
1

Understanding and interpreting a 3d environment is a key challenge for autonomous vehicles. Semantic segmentation of 3d point clouds combines 3d information with semantics and thereby provides a valuable contribution to this task. In many real-world applications, point clouds are generated by lidar sensors in a consecutive fashion. Working with a time series instead of single and independent frames enables the exploitation of temporal information. We therefore propose a recurrent segmentation architecture (RNN), which takes a single range image frame as input and exploits recursively aggregated temporal information. An alignment strategy, which we call Temporal Memory Alignment, uses ego motion to temporally align the memory between consecutive frames in feature space. A Residual Network and ConvGRU are investigated for the memory update. We demonstrate the benefits of the presented approach on two large-scale datasets and compare it to several stateof-the-art methods. Our approach ranks first on the SemanticKITTI multiple scan benchmark and achieves state-of-the-art performance on the single scan benchmark. In addition, the evaluation shows that the exploitation of temporal information significantly improves segmentation results compared to a single frame approach.

READ FULL TEXT
research
07/18/2023

MarS3D: A Plug-and-Play Motion-Aware Model for Semantic Segmentation on Multi-Scan 3D Point Clouds

3D semantic segmentation on multi-scan large-scale point clouds plays an...
research
08/03/2020

Frame-To-Frame Consistent Semantic Segmentation

In this work, we aim for temporally consistent semantic segmentation thr...
research
02/27/2022

Meta-RangeSeg: LiDAR Sequence Semantic Segmentation Using Multiple Feature Aggregation

LiDAR sensor is essential to the perception system in autonomous vehicle...
research
10/14/2022

LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds

Semantic segmentation of LiDAR point clouds is an important task in auto...
research
08/25/2023

SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation

LiDAR-based semantic perception tasks are critical yet challenging for a...
research
12/07/2022

Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data

Scene understanding is crucial for autonomous robots in dynamic environm...
research
06/30/2020

Leveraging Temporal Information for 3D Detection and Domain Adaptation

Ever since the prevalent use of the LiDARs in autonomous driving, tremen...

Please sign up or login with your details

Forgot password? Click here to reset