Log In Sign Up

Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration

by   Yuhang He, et al.

In this paper, we focus on the multi-object tracking (MOT) problem of automatic driving and robot navigation. Most existing MOT methods track multiple objects using a singular RGB camera, which are prone to camera field-of-view and suffer tracking failures in complex scenarios due to background clutters and poor light conditions. To meet these challenges, we propose a MultiModality PAnoramic multi-object Tracking framework (MMPAT), which takes both 2D panorama images and 3D point clouds as input and then infers target trajectories using the multimodality data. The proposed method contains four major modules, a panorama image detection module, a multimodality data fusion module, a data association module and a trajectory inference model. We evaluate the proposed method on the JRDB dataset, where the MMPAT achieves the top performance in both the detection and tracking tasks and significantly outperforms state-of-the-art methods by a large margin (15.7 and 8.5 improvement in terms of AP and MOTA, respectively).


page 1

page 3

page 4

page 8


An equalised global graphical model-based approach for multi-camera object tracking

Non-overlapping multi-camera visual object tracking typically consists o...

Mono-Camera 3D Multi-Object Tracking Using Deep Learning Detections and PMBM Filtering

Monocular cameras are one of the most commonly used sensors in the autom...

Detection and Tracking of Multiple Mice Using Part Proposal Networks

The study of mouse social behaviours has been increasingly undertaken in...

Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking

This paper introduces geometry and object shape and pose costs for multi...

DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association

In the recent literature, on the one hand, many 3D multi-object tracking...

Multi-Drone based Single Object Tracking with Agent Sharing Network

Drone equipped with cameras can dynamically track the target in the air ...

Multiple Object Tracking with Mixture Density Networks for Trajectory Estimation

Multiple object tracking faces several challenges that may be alleviated...