Action Recognition with Dynamic Image Networks

12/02/2016
by   Hakan Bilen, et al.
0

We introduce the concept of "dynamic image", a novel compact representation of videos useful for video analysis, particularly in combination with convolutional neural networks (CNNs). A dynamic image encodes temporal data such as RGB or optical flow videos by using the concept of `rank pooling'. The idea is to learn a ranking machine that captures the temporal evolution of the data and to use the parameters of the latter as a representation. When a linear ranking machine is used, the resulting representation is in the form of an image, which we call dynamic because it summarizes the video dynamics in addition of appearance. This is a powerful idea because it allows to convert any video to an image so that existing CNN models pre-trained for the analysis of still images can be immediately extended to videos. We also present an efficient and effective approximate rank pooling operator, accelerating standard rank pooling algorithms by orders of magnitude, and formulate that as a CNN layer. This new layer allows generalizing dynamic images to dynamic feature maps. We demonstrate the power of the new representations on standard benchmarks in action recognition achieving state-of-the-art performance.

READ FULL TEXT

page 1

page 4

page 7

page 8

page 10

page 11

page 14

research
12/06/2015

Rank Pooling for Action Recognition

We propose a function-based temporal pooling method that captures the la...
research
01/21/2019

Semantic Image Networks for Human Action Recognition

In this paper, we propose the use of a semantic image, an improved repre...
research
01/12/2017

Ordered Pooling of Optical Flow Sequences for Action Recognition

Training of Convolutional Neural Networks (CNNs) on long video sequences...
research
05/24/2017

Sequence Summarization Using Order-constrained Kernelized Feature Subspaces

Representations that can compactly and effectively capture temporal evol...
research
03/27/2018

Non-Linear Temporal Subspace Representations for Activity Recognition

Representations that can compactly and effectively capture the temporal ...
research
10/27/2021

Temporal-attentive Covariance Pooling Networks for Video Recognition

For video recognition task, a global representation summarizing the whol...
research
11/21/2016

Deep Temporal Linear Encoding Networks

The CNN-encoding of features from entire videos for the representation o...

Please sign up or login with your details

Forgot password? Click here to reset