DeepAI
Log In Sign Up

On the Integration of Optical Flow and Action Recognition

12/22/2017
by   Laura Sevilla-Lara, et al.
0

Most of the top performing action recognition methods use optical flow as a "black box" input. Here we take a deeper look at the combination of flow and action recognition, and investigate why optical flow is helpful, what makes a flow method good for action recognition, and how we can make it better. In particular, we investigate the impact of different flow algorithms and input transformations to better understand how these affect a state-of-the-art action recognition method. Furthermore, we fine tune two neural-network flow methods end-to-end on the most widely used action recognition dataset (UCF101). Based on these experiments, we make the following five observations: 1) optical flow is useful for action recognition because it is invariant to appearance, 2) optical flow methods are optimized to minimize end-point-error (EPE), but the EPE of current methods is not well correlated with action recognition performance, 3) for the flow methods tested, accuracy at boundaries and at small displacements is most correlated with action recognition performance, 4) training optical flow to minimize classification error instead of minimizing EPE improves recognition performance, and 5) optical flow learned for the task of action recognition differs from traditional optical flow especially inside the human body and at the boundary of the body. These observations may encourage optical flow researchers to look beyond EPE as a goal and guide action recognition researchers to seek better motion cues, leading to a tighter integration of the optical flow and action recognition communities.

READ FULL TEXT

page 2

page 4

page 8

10/01/2013

Combining Spatio-Temporal Appearance Descriptors and Optical Flow for Human Action Recognition in Video Data

This paper proposes combining spatio-temporal appearance (STA) descripto...
12/09/2016

ActionFlowNet: Learning Motion Representation for Action Recognition

Even with the recent advances in convolutional neural networks (CNN) in ...
10/02/2018

Representation Flow for Action Recognition

In this paper, we propose a convolutional layer inspired by optical flow...
03/05/2021

Unsupervised Motion Representation Enhanced Network for Action Recognition

Learning reliable motion representation between consecutive frames, such...
12/22/2018

Temporal Hockey Action Recognition via Pose and Optical Flows

Recognizing actions in ice hockey using computer vision poses challenges...
09/02/2014

Action Recognition in the Frequency Domain

In this paper, we describe a simple strategy for mitigating variability ...
08/08/2020

PAN: Towards Fast Action Recognition via Learning Persistence of Appearance

Efficiently modeling dynamic motion information in videos is crucial for...