Mimetics: Towards Understanding Human Actions Out of Context

12/16/2019
by   Philippe Weinzaepfel, et al.
17

Recent methods for video action recognition have reached outstanding performances on existing benchmarks. However, they tend to leverage context such as scenes or objects instead of focusing on understanding the human action itself. For instance, a tennis field leads to the prediction playing tennis irrespectively of the actions performed in the video. In contrast, humans have a more complete understanding of actions and can recognize them without context. The best example of out-of-context actions are mimes, that people can typically recognize despite missing relevant objects and scenes. In this paper, we propose to benchmark action recognition methods in the absence of context. We therefore introduce a novel dataset, Mimetics, consisting of mimed actions for a subset of 50 classes from the Kinetics benchmark. Our experiments show that state-of-the-art 3D convolutional neural networks obtain disappointing results on such videos, highlighting the lack of true understanding of the human actions. Body language, captured by human pose and motion, is a meaningful cue to recognize out-of-context actions. We thus evaluate several pose-based baselines, either based on explicit 2D or 3D pose estimates, or on transferring pose features to the action recognition problem. This last method, less prone to inherent pose estimation noise, performs better than the other pose-based baselines, suggesting that an explicit pose representation might not be optimal for real-world action recognition.

READ FULL TEXT

page 1

page 2

page 8

page 12

research
10/17/2020

A Grid-based Representation for Human Action Recognition

Human action recognition (HAR) in videos is a fundamental research topic...
research
05/22/2018

Pose-Based Two-Stream Relational Networks for Action Recognition in Videos

Recently, pose-based action recognition has gained more and more attenti...
research
04/19/2019

Simple yet efficient real-time pose-based action recognition

Recognizing human actions is a core challenge for autonomous systems as ...
research
05/01/2018

Object Activity Scene Description, Construction and Recognition

Action recognition is a critical task for social robots to meaningfully ...
research
11/01/2022

No-audio speaking status detection in crowded settings via visual pose-based filtering and wearable acceleration

Recognizing who is speaking in a crowded scene is a key challenge toward...
research
04/19/2022

ActAR: Actor-Driven Pose Embeddings for Video Action Recognition

Human action recognition (HAR) in videos is one of the core tasks of vid...
research
01/17/2016

Face-space Action Recognition by Face-Object Interactions

Action recognition in still images has seen major improvement in recent ...

Please sign up or login with your details

Forgot password? Click here to reset