DeepAI AI Chat
Log In Sign Up

MGSampler: An Explainable Sampling Strategy for Video Action Recognition

by   Yuan Zhi, et al.

Frame sampling is a fundamental problem in video action recognition due to the essential redundancy in time and limited computation resources. The existing sampling strategy often employs a fixed frame selection and lacks the flexibility to deal with complex variations in videos. In this paper, we present an explainable, adaptive, and effective frame sampler, called Motion-guided Sampler (MGSampler). Our basic motivation is that motion is an important and universal signal that can drive us to select frames from videos adaptively. Accordingly, we propose two important properties in our MGSampler design: motion sensitive and motion uniform. First, we present two different motion representations to enable us to efficiently distinguish the motion salient frames from the background. Then, we devise a motion-uniform sampling strategy based on the cumulative motion distribution to ensure the sampled frames evenly cover all the important frames with high motion saliency. Our MGSampler yields a new principled and holistic sample scheme, that could be incorporated into any existing video architecture. Experiments on five benchmarks demonstrate the effectiveness of our MGSampler over previously fixed sampling strategies, and also its generalization power across different backbones, video models, and datasets.


page 2

page 3

page 5


PMI Sampler: Patch similarity guided frame selection for Aerial Action Recognition

We present a new algorithm for selection of informative frames in video ...

OCSampler: Compressing Videos to One Clip with Single-step Sampling

In this paper, we propose a framework named OCSampler to explore a compa...

Task-adaptive Spatial-Temporal Video Sampler for Few-shot Action Recognition

A primary challenge faced in few-shot action recognition is inadequate v...

Random Temporal Skipping for Multirate Video Analysis

Current state-of-the-art approaches to video understanding adopt tempora...

Challenge report:VIPriors Action Recognition Challenge

This paper is a brief report to our submission to the VIPriors Action Re...

Adversarially Robust Frame Sampling with Bounded Irregularities

In recent years, video analysis tools for automatically extracting meani...

MotionSqueeze: Neural Motion Feature Learning for Video Understanding

Motion plays a crucial role in understanding videos and most state-of-th...