DeepAI AI Chat
Log In Sign Up

Concurrence-Aware Long Short-Term Sub-Memories for Person-Person Action Recognition

by   Xiangbo Shu, et al.

Recently, Long Short-Term Memory (LSTM) has become a popular choice to model individual dynamics for single-person action recognition due to its ability of modeling the temporal information in various ranges of dynamic contexts. However, existing RNN models only focus on capturing the temporal dynamics of the person-person interactions by naively combining the activity dynamics of individuals or modeling them as a whole. This neglects the inter-related dynamics of how person-person interactions change over time. To this end, we propose a novel Concurrence-Aware Long Short-Term Sub-Memories (Co-LSTSM) to model the long-term inter-related dynamics between two interacting people on the bounding boxes covering people. Specifically, for each frame, two sub-memory units store individual motion information, while a concurrent LSTM unit selectively integrates and stores inter-related motion information between interacting people from these two sub-memory units via a new co-memory cell. Experimental results on the BIT and UT datasets show the superiority of Co-LSTSM compared with the state-of-the-art methods.


page 1

page 2

page 3

page 4


Hierarchical Long Short-Term Concurrent Memory for Human Interaction Recognition

In this paper, we aim to address the problem of human interaction recogn...

Lattice Long Short-Term Memory for Human Action Recognition

Human actions captured in video sequences are three-dimensional signals ...

Making Third Person Techniques Recognize First-Person Actions in Egocentric Videos

We focus on first-person action recognition from egocentric videos. Unli...

Listen to Look: Action Recognition by Previewing Audio

In the face of the video data deluge, today's expensive clip-level class...

LSTC: Boosting Atomic Action Detection with Long-Short-Term Context

In this paper, we place the atomic action detection problem into a Long-...

Tweets Can Tell: Activity Recognition using Hybrid Long Short-Term Memory Model

This paper presents techniques to detect the "offline" activity a person...

Learning of Multi-Context Models for Autonomous Underwater Vehicles

Multi-context model learning is crucial for marine robotics where severa...