Deep Action- and Context-Aware Sequence Learning for Activity Recognition and Anticipation

Action recognition and anticipation are key to the success of many computer vision applications. Existing methods can roughly be grouped into those that extract global, context-aware representations of the entire image or sequence, and those that aim at focusing on the regions where the action occurs. While the former may suffer from the fact that context is not always reliable, the latter completely ignore this source of information, which can nonetheless be helpful in many situations. In this paper, we aim at making the best of both worlds by developing an approach that leverages both context-aware and action-aware features. At the core of our method lies a novel multi-stage recurrent architecture that allows us to effectively combine these two sources of information throughout a video. This architecture first exploits the global, context-aware features, and merges the resulting representation with the localized, action-aware ones. Our experiments on standard datasets evidence the benefits of our approach over methods that use each information type separately. We outperform the state-of-the-art methods that, as us, rely only on RGB frames as input for both action recognition and anticipation.


page 1

page 2

page 3

page 4


Encouraging LSTMs to Anticipate Actions Very Early

In contrast to the widely studied problem of recognizing an action given...

Skeleton Based Human Action Recognition with Global Context-Aware Attention LSTM Networks

Human action recognition in 3D skeleton sequences has attracted a lot of...

CAPHAR: context-aware personalized human activity recognition using associative learning in smart environments

The existing action recognition systems mainly focus on generalized meth...

Neuro-Symbolic Approaches for Context-Aware Human Activity Recognition

Deep Learning models are a standard solution for sensor-based Human Acti...

Context-aware Automatic Music Transcription

This paper presents an Automatic Music Transcription system that incorpo...

Cyclone intensity estimate with context-aware cyclegan

Deep learning approaches to cyclone intensity estimationhave recently sh...

Learning to Discriminate Information for Online Action Detection: Analysis and Application

Online action detection, which aims to identify an ongoing action from a...

Please sign up or login with your details

Forgot password? Click here to reset