Synthetic Data Are as Good as the Real for Association Knowledge Learning in Multi-object Tracking

06/30/2021
by   Yuchi Liu, et al.
0

Association, aiming to link bounding boxes of the same identity in a video sequence, is a central component in multi-object tracking (MOT). To train association modules, e.g., parametric networks, real video data are usually used. However, annotating person tracks in consecutive video frames is expensive, and such real data, due to its inflexibility, offer us limited opportunities to evaluate the system performance w.r.t changing tracking scenarios. In this paper, we study whether 3D synthetic data can replace real-world videos for association training. Specifically, we introduce a large-scale synthetic data engine named MOTX, where the motion characteristics of cameras and objects are manually configured to be similar to those in real-world datasets. We show that compared with real data, association knowledge obtained from synthetic data can achieve very similar performance on real-world test sets without domain adaption techniques. Our intriguing observation is credited to two factors. First and foremost, 3D engines can well simulate motion factors such as camera movement, camera view and object movement, so that the simulated videos can provide association modules with effective motion features. Second, experimental results show that the appearance domain gap hardly harms the learning of association knowledge. In addition, the strong customization ability of MOTX allows us to quantitatively assess the impact of motion factors on MOT, which brings new insights to the community.

READ FULL TEXT
research
04/11/2023

SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes

Multi-object tracking in sports scenes plays a critical role in gatherin...
research
11/07/2022

TAP-Vid: A Benchmark for Tracking Any Point in a Video

Generic motion understanding from video involves not only tracking objec...
research
03/27/2023

ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box

Multi-object tracking (MOT) aims at estimating bounding boxes and identi...
research
09/10/2018

Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackers

Online Multi-Object Tracking (MOT) from videos is a challenging computer...
research
06/10/2020

Map3D: Registration Based Multi-Object Tracking on 3D Serial Whole Slide Images

There has been a long pursuit for precise and reproducible glomerular qu...
research
03/26/2023

SDTracker: Synthetic Data Based Multi-Object Tracking

We present SDTracker, a method that harnesses the potential of synthetic...
research
06/12/2021

DyGLIP: A Dynamic Graph Model with Link Prediction for Accurate Multi-Camera Multiple Object Tracking

Multi-Camera Multiple Object Tracking (MC-MOT) is a significant computer...

Please sign up or login with your details

Forgot password? Click here to reset