Self-Supervised Multi-Object Tracking with Cross-Input Consistency

11/10/2021
by   Favyen Bastani, et al.
0

In this paper, we propose a self-supervised learning procedure for training a robust multi-object tracking (MOT) model given only unlabeled video. While several self-supervisory learning signals have been proposed in prior work on single-object tracking, such as color propagation and cycle-consistency, these signals cannot be directly applied for training RNN models, which are needed to achieve accurate MOT: they yield degenerate models that, for instance, always match new detections to tracks with the closest initial detections. We propose a novel self-supervisory signal that we call cross-input consistency: we construct two distinct inputs for the same sequence of video, by hiding different information about the sequence in each input. We then compute tracks in that sequence by applying an RNN model independently on each input, and train the model to produce consistent tracks across the two inputs. We evaluate our unsupervised method on MOT17 and KITTI – remarkably, we find that, despite training only on unlabeled video, our unsupervised approach outperforms four supervised methods published in the last 1–2 years, including Tracktor++, FAMNet, GSM, and mmMOT.

READ FULL TEXT
research
01/26/2023

Self-Supervised RGB-T Tracking with Cross-Input Consistency

In this paper, we propose a self-supervised RGB-T tracking method. Diffe...
research
10/26/2020

Multi-object tracking with self-supervised associating network

Multi-Object Tracking (MOT) is the task that has a lot of potential for ...
research
05/17/2023

S^3Track: Self-supervised Tracking with Soft Assignment Flow

In this work, we study self-supervised multiple object tracking without ...
research
09/02/2023

Tracking without Label: Unsupervised Multiple Object Tracking via Contrastive Similarity Learning

Unsupervised learning is a challenging task due to the lack of labels. M...
research
05/20/2021

Robust Unsupervised Multi-Object Tracking in Noisy Environments

Physical processes, camera movement, and unpredictable environmental con...
research
06/07/2022

Online Deep Clustering with Video Track Consistency

Several unsupervised and self-supervised approaches have been developed ...
research
07/28/2023

Uncertainty-aware Unsupervised Multi-Object Tracking

Without manually annotated identities, unsupervised multi-object tracker...

Please sign up or login with your details

Forgot password? Click here to reset