IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos

07/13/2020
by   Gyeongsik Moon, et al.
0

Most current action recognition methods heavily rely on appearance information by taking an RGB sequence of entire image regions as input. While being effective in exploiting contextual information around humans, e.g., human appearance and scene category, they are easily fooled by out-of-context action videos where the contexts do not exactly match with target actions. In contrast, pose-based methods, which takes a sequence of human skeletons only as input, suffer from inaccurate pose estimation or ambiguity of human pose per se. Integrating these two approaches has turned out to be non-trivial; training a model with both appearance and pose ends up with a strong bias towards appearance and does not generalize well to unseen videos. To address this problem, we propose to learn pose-driven feature integration that dynamically combines appearance and pose streams by observing pose features on the fly. The main idea is to let the pose stream decide how much and which appearance information is used in integration based on whether the given pose information is reliable or not. We show that the proposed IntegralAction achieves highly robust performance across in-context and out-of-context action video datasets.

READ FULL TEXT

page 1

page 16

page 17

page 18

page 19

research
05/22/2018

Pose-Based Two-Stream Relational Networks for Action Recognition in Videos

Recently, pose-based action recognition has gained more and more attenti...
research
03/13/2016

Pose for Action - Action for Pose

In this work we propose to utilize information about human actions to im...
research
09/28/2020

PERF-Net: Pose Empowered RGB-Flow Net

In recent years, many works in the video action recognition literature h...
research
05/20/2020

Discriminative Dictionary Design for Action Classification in Still Images and Videos

In this paper, we address the problem of action recognition from still i...
research
10/26/2020

Handgun detection using combined human pose and weapon appearance

CCTV surveillance systems are essential nowadays to prevent and mitigate...
research
10/16/2020

Pose And Joint-Aware Action Recognition

Most human action recognition systems typically consider static appearan...
research
12/25/2018

Coupled Recurrent Network (CRN)

Many semantic video analysis tasks can benefit from multiple, heterogeno...

Please sign up or login with your details

Forgot password? Click here to reset