Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection

03/21/2023

∙

by   Shihao Wang, et al.

∙

8

∙

share

In this paper, we propose a long-sequence modeling framework, named StreamPETR, for multi-view 3D object detection. Built upon the sparse query design in the PETR series, we systematically develop an object-centric temporal mechanism. The model is performed in an online manner and the long-term historical information is propagated through object queries frame by frame. Besides, we introduce a motion-aware layer normalization to model the movement of the objects. StreamPETR achieves significant performance improvements only with negligible computation cost, compared to the single-frame baseline. On the standard nuScenes benchmark, it reaches a new state-of-the-art performance (63.6 outperforming the state-of-the-art method (SOLOFusion) by 2.3 faster FPS. Code will be available at https://github.com/exiawsh/StreamPETR.git.