MGSampler: An Explainable Sampling Strategy for Video Action Recognition

04/20/2021
by   Yuan Zhi, et al.
0

Frame sampling is a fundamental problem in video action recognition due to the essential redundancy in time and limited computation resources. The existing sampling strategy often employs a fixed frame selection and lacks the flexibility to deal with complex variations in videos. In this paper, we present an explainable, adaptive, and effective frame sampler, called Motion-guided Sampler (MGSampler). Our basic motivation is that motion is an important and universal signal that can drive us to select frames from videos adaptively. Accordingly, we propose two important properties in our MGSampler design: motion sensitive and motion uniform. First, we present two different motion representations to enable us to efficiently distinguish the motion salient frames from the background. Then, we devise a motion-uniform sampling strategy based on the cumulative motion distribution to ensure the sampled frames evenly cover all the important frames with high motion saliency. Our MGSampler yields a new principled and holistic sample scheme, that could be incorporated into any existing video architecture. Experiments on five benchmarks demonstrate the effectiveness of our MGSampler over previously fixed sampling strategies, and also its generalization power across different backbones, video models, and datasets.

READ FULL TEXT

Authors

page 2

page 3

page 5

10/30/2018

Random Temporal Skipping for Multirate Video Analysis

Current state-of-the-art approaches to video understanding adopt tempora...
01/12/2022

OCSampler: Compressing Videos to One Clip with Single-step Sampling

In this paper, we propose a framework named OCSampler to explore a compa...
12/19/2020

SMART Frame Selection for Action Recognition

Action recognition is computationally expensive. In this paper, we addre...
06/02/2021

TSI: Temporal Saliency Integration for Video Action Recognition

Efficient spatiotemporal modeling is an important yet challenging proble...
07/16/2020

Challenge report:VIPriors Action Recognition Challenge

This paper is a brief report to our submission to the VIPriors Action Re...
02/04/2020

Adversarially Robust Frame Sampling with Bounded Irregularities

In recent years, video analysis tools for automatically extracting meani...
12/11/2014

EgoSampling: Fast-Forward and Stereo for Egocentric Videos

While egocentric cameras like GoPro are gaining popularity, the videos t...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.