Object Detection in Videos with Tubelet Proposal Networks

by   Kai Kang, et al.
SenseTime Corporation
Tsinghua University
The Chinese University of Hong Kong

Object detection in videos has drawn increasing attention recently with the introduction of the large-scale ImageNet VID dataset. Different from object detection in static images, temporal information in videos is vital for object detection. To fully utilize temporal information, state-of-the-art methods are based on spatiotemporal tubelets, which are essentially sequences of associated bounding boxes across time. However, the existing methods have major limitations in generating tubelets in terms of quality and efficiency. Motion-based methods are able to obtain dense tubelets efficiently, but the lengths are generally only several frames, which is not optimal for incorporating long-term temporal information. Appearance-based methods, usually involving generic object tracking, could generate long tubelets, but are usually computationally expensive. In this work, we propose a framework for object detection in videos, which consists of a novel tubelet proposal network to efficiently generate spatiotemporal proposals, and a Long Short-term Memory (LSTM) network that incorporates temporal information from tubelet proposals for achieving high object detection accuracy in videos. Experiments on the large-scale ImageNet VID dataset demonstrate the effectiveness of the proposed framework for object detection in videos.


page 1

page 5

page 8


Tube-CNN: Modeling temporal evolution of appearance for object detection in video

Object detection in video is crucial for many applications. Compared to ...

A Generic Object Re-identification System for Short Videos

Short video applications like TikTok and Kwai have been a great hit rece...

Temporal Dynamic Graph LSTM for Action-driven Video Object Detection

In this paper, we investigate a weakly-supervised object detection frame...

STAU: A SpatioTemporal-Aware Unit for Video Prediction and Beyond

Video prediction aims to predict future frames by modeling the complex s...

Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds

Previous works for LiDAR-based 3D object detection mainly focus on the s...

Recurrent Neural Networks for video object detection

There is lots of scientific work about object detection in images. For m...

TSSD: Temporal Single-Shot Object Detection Based on Attention-Aware LSTM

Temporal object detection has attracted significant attention, but most ...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset