SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videos

by   Haisong Liu, et al.
Nanjing University

Camera-based 3D object detection in BEV (Bird's Eye View) space has drawn great attention over the past few years. Dense detectors typically follow a two-stage pipeline by first constructing a dense BEV feature and then performing object detection in BEV space, which suffers from complex view transformations and high computation cost. On the other side, sparse detectors follow a query-based paradigm without explicit dense BEV feature construction, but achieve worse performance than the dense counterparts. In this paper, we find that the key to mitigate this performance gap is the adaptability of the detector in both BEV and image space. To achieve this goal, we propose SparseBEV, a fully sparse 3D object detector that outperforms the dense counterparts. SparseBEV contains three key designs, which are (1) scale-adaptive self attention to aggregate features with adaptive receptive field in BEV space, (2) adaptive spatio-temporal sampling to generate sampling locations under the guidance of queries, and (3) adaptive mixing to decode the sampled features with dynamic weights from the queries. On the test split of nuScenes, SparseBEV achieves the state-of-the-art performance of 67.5 NDS. On the val split, SparseBEV achieves 55.8 NDS while maintaining a real-time inference speed of 23.5 FPS. Code is available at


page 3

page 8

page 12

page 14


Object as Query: Equipping Any 2D Object Detector with 3D Detection Ability

3D object detection from multi-view images has drawn much attention over...

ASAG: Building Strong One-Decoder-Layer Sparse Detectors via Adaptive Sparse Anchor Generation

Recent sparse detectors with multiple, e.g. six, decoder layers achieve ...

QueryProp: Object Query Propagation for High-Performance Video Object Detection

Video object detection has been an important yet challenging topic in co...

ConQueR: Query Contrast Voxel-DETR for 3D Object Detection

Although DETR-based 3D detectors can simplify the detection pipeline and...

AdaMixer: A Fast-Converging Query-Based Object Detector

Traditional object detectors employ the dense paradigm of scanning over ...

SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection

By identifying four important components of existing LiDAR-camera 3D obj...

Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3D Object Detection

The dominant multi-camera 3D detection paradigm is based on explicit 3D ...

Please sign up or login with your details

Forgot password? Click here to reset