Hidden Two-Stream Convolutional Networks for Action Recognition

04/02/2017
by   Yi Zhu, et al.
0

Analyzing videos of human actions involves understanding the temporal relationships among video frames. CNNs are the current state-of-the-art methods for action recognition in videos. However, the CNN architectures currently being used have difficulty in capturing these relationships. State-of-the-art action recognition approaches rely on traditional local optical flow estimation methods to pre-compute motion information for CNNs. Such a two-stage approach is computationally expensive, storage demanding, and not end-to-end trainable. In this paper, we present a novel CNN architecture that implicitly captures motion information between adjacent frames. We then plug it into a state-of-the-art action recognition framework called two-stream CNNs. We name our approach hidden two-stream CNNs because it only takes raw video frames as input and directly predicts action classes without explicitly computing optical flow. Our end-to-end approach is 10x faster than a two-stage one and maintains similar accuracy. Experimental results on UCF101 and HMDB51 datasets show that our approach significantly outperforms previous best real-time approaches.

READ FULL TEXT
research
04/26/2016

Real-time Action Recognition with Enhanced Motion Vector CNNs

The deep two-stream architecture exhibited excellent performance on vide...
research
10/30/2018

Random Temporal Skipping for Multirate Video Analysis

Current state-of-the-art approaches to video understanding adopt tempora...
research
06/07/2019

Video Modeling with Correlation Networks

Motion is a salient cue to recognize actions in video. Modern action rec...
research
05/18/2017

Learning Spatiotemporal Features for Infrared Action Recognition with 3D Convolutional Neural Networks

Infrared (IR) imaging has the potential to enable more robust action rec...
research
10/17/2021

TEAM-Net: Multi-modal Learning for Video Action Recognition with Partial Decoding

Most of existing video action recognition models ingest raw RGB frames. ...
research
05/25/2019

Exploring Temporal Information for Improved Video Understanding

In this dissertation, I present my work towards exploring temporal infor...
research
06/13/2019

Hallucinating Bag-of-Words and Fisher Vector IDT terms for CNN-based Action Recognition

In this paper, we revive the use of old-fashioned handcrafted video repr...

Please sign up or login with your details

Forgot password? Click here to reset