MITFAS: Mutual Information based Temporal Feature Alignment and Sampling for Aerial Video Action Recognition

03/05/2023
by   Ruiqi Xian, et al.
0

We present a novel approach for action recognition in UAV videos. Our formulation is designed to handle occlusion and viewpoint changes caused by the movement of a UAV. We use the concept of mutual information to compute and align the regions corresponding to human action or motion in the temporal domain. This enables our recognition model to learn from the key features associated with the motion. We also propose a novel frame sampling method that uses joint mutual information to acquire the most informative frame sequence in UAV videos. We have integrated our approach with X3D and evaluated the performance on multiple datasets. In practice, we achieve 18.9 Top-1 accuracy over current state-of-the-art methods on UAV-Human(Li et al., 2021), 7.3 improvement on NEC Drones(Choi et al., 2020). We will release the code at the time of publication

READ FULL TEXT

page 1

page 3

research
04/14/2023

PMI Sampler: Patch similarity guided frame selection for Aerial Action Recognition

We present a new algorithm for selection of informative frames in video ...
research
05/10/2023

Few-shot Action Recognition via Intra- and Inter-Video Information Maximization

Current few-shot action recognition involves two primary sources of info...
research
09/15/2022

Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition

We present a learning algorithm for human activity recognition in videos...
research
06/19/2012

Dependence Maximizing Temporal Alignment via Squared-Loss Mutual Information

The goal of temporal alignment is to establish time correspondence betwe...
research
03/02/2023

AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning

We propose a novel approach for aerial video action recognition. Our met...
research
03/21/2022

Fourier Disentangled Space-Time Attention for Aerial Video Recognition

We present an algorithm, Fourier Activity Recognition (FAR), for UAV vid...
research
05/21/2023

Prompt Learning for Action Recognition

We present a new general learning approach for action recognition, Promp...

Please sign up or login with your details

Forgot password? Click here to reset