4D Panoptic Segmentation as Invariant and Equivariant Field Prediction

by   Minghan Zhu, et al.

In this paper, we develop rotation-equivariant neural networks for 4D panoptic segmentation. 4D panoptic segmentation is a recently established benchmark task for autonomous driving, which requires recognizing semantic classes and object instances on the road based on LiDAR scans, as well as assigning temporally consistent IDs to instances across time. We observe that the driving scenario is symmetric to rotations on the ground plane. Therefore, rotation-equivariance could provide better generalization and more robust feature learning. Specifically, we review the object instance clustering strategies, and restate the centerness-based approach and the offset-based approach as the prediction of invariant scalar fields and equivariant vector fields. Other sub-tasks are also unified from this perspective, and different invariant and equivariant layers are designed to facilitate their predictions. Through evaluation on the standard 4D panoptic segmentation benchmark of SemanticKITTI, we show that our equivariant models achieve higher accuracy with lower computational costs compared to their non-equivariant counterparts. Moreover, our method sets the new state-of-the-art performance and achieves 1st place on the SemanticKITTI 4D Panoptic Segmentation leaderboard.


SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clustering

Panoptic segmentation aims to address semantic and instance segmentation...

LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network

With the rapid advances of autonomous driving, it becomes critical to eq...

LiDAR-based Panoptic Segmentation via Dynamic Shifting Network

With the rapid advances of autonomous driving, it becomes critical to eq...

DeepTemporalSeg: Temporally Consistent Semantic Segmentation of 3D LiDAR Scans

Understanding the semantic characteristics of the environment is a key e...

DuEqNet: Dual-Equivariance Network in Outdoor 3D Object Detection for Autonomous Driving

Outdoor 3D object detection has played an essential role in the environm...

Orientation-aware Semantic Segmentation on Icosahedron Spheres

We address semantic segmentation on omnidirectional images, to leverage ...

SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation

State-of-the-art semantic or instance segmentation deep neural networks ...

Please sign up or login with your details

Forgot password? Click here to reset