DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association

by   Xiyang Wang, et al.

In the recent literature, on the one hand, many 3D multi-object tracking (MOT) works have focused on tracking accuracy and neglected computation speed, commonly by designing rather complex cost functions and feature extractors. On the other hand, some methods have focused too much on computation speed at the expense of tracking accuracy. In view of these issues, this paper proposes a robust and fast camera-LiDAR fusion-based MOT method that achieves a good trade-off between accuracy and speed. Relying on the characteristics of camera and LiDAR sensors, an effective deep association mechanism is designed and embedded in the proposed MOT method. This association mechanism realizes tracking of an object in a 2D domain when the object is far away and only detected by the camera, and updating of the 2D trajectory with 3D information obtained when the object appears in the LiDAR field of view to achieve a smooth fusion of 2D and 3D trajectories. Extensive experiments based on the KITTI dataset indicate that our proposed method presents obvious advantages over the state-of-the-art MOT methods in terms of both tracking accuracy and processing speed. Our code is made publicly available for the benefit of the community


page 1

page 6

page 7


Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

Multi-object tracking (MOT) with camera-LiDAR fusion demands accurate re...

CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion

3D Multi-object tracking (MOT) ensures consistency during continuous dyn...

Ego-Downward and Ambient Video based Person Location Association

Using an ego-centric camera to do localization and tracking is highly ne...

Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration

In this paper, we focus on the multi-object tracking (MOT) problem of au...

AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics

3D multi-object tracking (MOT) is essential to applications such as auto...

3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention

Online 3D multi-object tracking (MOT) has witnessed significant research...

Distributed Multi-object Tracking under Limited Field of View Sensors

We consider the challenging problem of tracking multiple objects using a...

Please sign up or login with your details

Forgot password? Click here to reset