Feature Flow: In-network Feature Flow Estimation for Video Object Detection

09/21/2020
by   Ruibing Jin, et al.
2

Optical flow, which expresses pixel displacement, is widely used in many computer vision tasks to provide pixel-level motion information. However, with the remarkable progress of the convolutional neural network, recent state-of-the-art approaches are proposed to solve problems directly on feature-level. Since the displacement of feature vector is not consistent to the pixel displacement, a common approach is to:forward optical flow to a neural network and fine-tune this network on the task dataset. With this method,they expect the fine-tuned network to produce tensors encoding feature-level motion information. In this paper, we rethink this de facto paradigm and analyze its drawbacks in the video object detection task. To mitigate these issues, we propose a novel network (IFF-Net) with an In-network Feature Flow estimation module (IFF module) for video object detection. Without resorting pre-training on any additional dataset, our IFF module is able to directly produce feature flow which indicates the feature displacement. Our IFF module consists of a shallow module, which shares the features with the detection branches. This compact design enables our IFF-Net to accurately detect objects, while maintaining a fast inference speed. Furthermore, we propose a transformation residual loss (TRL) based on self-supervision, which further improves the performance of our IFF-Net. Our IFF-Net outperforms existing methods and sets a state-of-the-art performance on ImageNet VID.

READ FULL TEXT

page 1

page 7

page 8

research
09/20/2017

SegFlow: Joint Learning for Video Object Segmentation and Optical Flow

This paper proposes an end-to-end trainable network, SegFlow, for simult...
research
01/17/2020

FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Semi-supervised Optical Flow Estimation

Optical flow estimation is an important yet challenging problem in the f...
research
07/23/2021

Detail Preserving Residual Feature Pyramid Modules for Optical Flow

Feature pyramids and iterative refinement have recently led to great pro...
research
05/22/2018

A Convolutional Feature Map based Deep Network targeted towards Traffic Detection and Classification

This research mainly emphasizes on traffic detection thus essentially in...
research
02/08/2021

Analysis of Latent-Space Motion for Collaborative Intelligence

When the input to a deep neural network (DNN) is a video signal, a seque...
research
03/21/2019

Progressive Sparse Local Attention for Video object detection

Transferring image-based object detectors to domain of videos remains a ...
research
07/11/2022

Snow Mask Guided Adaptive Residual Network for Image Snow Removal

Image restoration under severe weather is a challenging task. Most of th...

Please sign up or login with your details

Forgot password? Click here to reset