DeepAI AI Chat
Log In Sign Up

Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

by   Kemiao Huang, et al.
Southern University of Science & Technology

Multi-object tracking (MOT) with camera-LiDAR fusion demands accurate results of object detection, affinity computation and data association in real time. This paper presents an efficient multi-modal MOT framework with online joint detection and tracking schemes and robust data association for autonomous driving applications. The novelty of this work includes: (1) development of an end-to-end deep neural network for joint object detection and correlation using 2D and 3D measurements; (2) development of a robust affinity computation module to compute occlusion-aware appearance and motion affinities in 3D space; (3) development of a comprehensive data association module for joint optimization among detection confidences, affinities and start-end probabilities. The experiment results on the KITTI tracking benchmark demonstrate the superior performance of the proposed method in terms of both tracking accuracy and processing speed.


page 1

page 6


CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking

3D multi-object tracking is a crucial component in the perception system...

DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association

In the recent literature, on the one hand, many 3D multi-object tracking...

CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion

3D Multi-object tracking (MOT) ensures consistency during continuous dyn...

GOOD: General Optimization-based Fusion for 3D Object Detection via LiDAR-Camera Object Candidates

3D object detection serves as the core basis of the perception tasks in ...

3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation

3D object detection using LiDAR data remains a key task for applications...

Fusion of Object Tracking and Dynamic Occupancy Grid Map

Environment modeling in autonomous driving is realized by two fundamenta...

Code Repositories


Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving

view repo