DynStatF: An Efficient Feature Fusion Strategy for LiDAR 3D Object Detection

05/24/2023
by   Yao Rong, et al.
0

Augmenting LiDAR input with multiple previous frames provides richer semantic information and thus boosts performance in 3D object detection, However, crowded point clouds in multi-frames can hurt the precise position information due to the motion blur and inaccurate point projection. In this work, we propose a novel feature fusion strategy, DynStaF (Dynamic-Static Fusion), which enhances the rich semantic information provided by the multi-frame (dynamic branch) with the accurate location information from the current single-frame (static branch). To effectively extract and aggregate complimentary features, DynStaF contains two modules, Neighborhood Cross Attention (NCA) and Dynamic-Static Interaction (DSI), operating through a dual pathway architecture. NCA takes the features in the static branch as queries and the features in the dynamic branch as keys (values). When computing the attention, we address the sparsity of point clouds and take only neighborhood positions into consideration. NCA fuses two features at different feature map scales, followed by DSI providing the comprehensive interaction. To analyze our proposed strategy DynStaF, we conduct extensive experiments on the nuScenes dataset. On the test set, DynStaF increases the performance of PointPillars in NDS by a large margin from 57.7 framework achieves 61.0 performance without bells and whistles.

READ FULL TEXT
research
03/30/2023

BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregation

Integrating LiDAR and Camera information into Bird's-Eye-View (BEV) has ...
research
09/16/2020

Dual Semantic Fusion Network for Video Object Detection

Video object detection is a tough task due to the deteriorated quality o...
research
11/03/2022

PointSee: Image Enhances Point Cloud

There is a trend to fuse multi-modal information for 3D object detection...
research
12/10/2022

Multi-Sem Fusion: Multimodal Semantic Fusion for 3D Object Detection

LiDAR-based 3D Object detectors have achieved impressive performances in...
research
09/15/2022

FFPA-Net: Efficient Feature Fusion with Projection Awareness for 3D Object Detection

Promising complementarity exists between the texture features of color i...
research
06/19/2023

Frame Fusion with Vehicle Motion Prediction for 3D Object Detection

In LiDAR-based 3D detection, history point clouds contain rich temporal ...
research
01/22/2023

Bidirectional Propagation for Cross-Modal 3D Object Detection

Recent works have revealed the superiority of feature-level fusion for c...

Please sign up or login with your details

Forgot password? Click here to reset