Video Classification with FineCoarse Networks

03/29/2021
by   Guoxi Huang, et al.
0

A rich representation of the information in video data can be realized by means of frequency analysis. Fine motion details from the boundaries of moving regions are characterized by high frequencies in the spatio-temporal domain. Meanwhile, lower frequencies are encoded with coarse information containing substantial redundancy, which causes low efficiency for those video models that take as input raw RGB frames. In this work, we propose a Motion Band-pass Module (MBPM) for separating the fine-grained information from coarse information in raw video data. By representing the coarse information with low resolution, we can increase the efficiency of video data processing. By embedding the MBPM into a two-pathway CNN architecture, we define a FineCoarse network. The efficiency of the FineCoarse network is determined by avoiding the redundancy in the feature space processed by the two pathways: one operates on downsampled features of low-resolution data, while the other operates on the fine-grained motion information captured by the MBPM. The proposed FineCoarse network outperforms many recent video processing models on Kinetics400, UCF101 and HMDB51. Furthermore, our approach achieves the state-of-the-art with 57.0 top-1 accuracy on Something-Something V1.

READ FULL TEXT

page 1

page 8

page 12

page 13

page 14

page 15

research
05/21/2016

Fine-to-coarse Knowledge Transfer For Low-Res Image Classification

We address the difficult problem of distinguishing fine-grained object c...
research
03/01/2021

Coarse-Fine Networks for Temporal Activity Detection in Videos

In this paper, we introduce 'Coarse-Fine Networks', a two-stream archite...
research
05/01/2021

Enhancing Fine-Grained Classification for Low Resolution Images

Low resolution fine-grained classification has widespread applicability ...
research
09/26/2022

Rethinking Resolution in the Context of Efficient Video Recognition

In this paper, we empirically study how to make the most of low-resoluti...
research
01/12/2022

Neural Residual Flow Fields for Efficient Video Representations

Implicit neural representation (INR) has emerged as a powerful paradigm ...
research
09/15/2023

Differentiable Resolution Compression and Alignment for Efficient Video Classification and Retrieval

Optimizing video inference efficiency has become increasingly important ...
research
11/07/2017

ZipNet-GAN: Inferring Fine-grained Mobile Traffic Patterns via a Generative Adversarial Neural Network

Large-scale mobile traffic analytics is becoming essential to digital in...

Please sign up or login with your details

Forgot password? Click here to reset