DeepAI AI Chat
Log In Sign Up

Detect to Track and Track to Detect

by   Christoph Feichtenhofer, et al.
University of Oxford
TU Graz

Recent approaches for high accuracy detection and tracking of object categories in video consist of complex multistage solutions that become more cumbersome each year. In this paper we propose a ConvNet architecture that jointly performs detection and tracking, solving the task in a simple and effective way. Our contributions are threefold: (i) we set up a ConvNet architecture for simultaneous detection and tracking, using a multi-task objective for frame-based object detection and across-frame track regression; (ii) we introduce correlation features that represent object co-occurrences across time to aid the ConvNet during tracking; and (iii) we link the frame level detections based on our across-frame tracklets to produce high accuracy detections at the video level. Our ConvNet architecture for spatiotemporal object detection is evaluated on the large-scale ImageNet VID dataset where it achieves state-of-the-art results. Our approach provides better single model performance than the winning method of the last ImageNet challenge while being conceptually much simpler. Finally, we show that by increasing the temporal stride we can dramatically increase the tracker speed.


page 1

page 5

page 6

page 9


Detect or Track: Towards Cost-Effective Video Object Detection/Tracking

State-of-the-art object detectors and trackers are developing fast. Trac...

Seq-NMS for Video Object Detection

Video object detection is challenging because objects that are easily de...

3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation

3D object detection using LiDAR data remains a key task for applications...

Disentangling What and Where for 3D Object-Centric Representations Through Active Inference

Although modern object detection and classification models achieve high ...

1st Place Solutions for the UVO Challenge 2022

This paper describes the approach we have taken in the challenge. We sti...

Video text tracking for dense and small text based on pp-yoloe-r and sort algorithm

Although end-to-end video text spotting methods based on Transformer can...

CoMaL Tracking: Tracking Points at the Object Boundaries

Traditional point tracking algorithms such as the KLT use local 2D infor...