Joint learning of Social Groups, Individuals Action and Sub-group Activities in Videos

07/06/2020
by   Mahsa Ehsanpour, et al.
4

The state-of-the art solutions for human activity understanding from a video stream formulate the task as a spatio-temporal problem which requires joint localization of all individuals in the scene and classification of their actions or group activity over time. Who is interacting with whom, e.g. not everyone in a queue is interacting with each other, is often not predicted. There are scenarios where people are best to be split into sub-groups, which we call social groups, and each social group may be engaged in a different social activity. In this paper, we solve the problem of simultaneously grouping people by their social interactions, predicting their individual actions and the social activity of each social group, which we call the social task. Our main contributions are: i) we propose an end-to-end trainable framework for the social task; ii) our proposed method also sets the state-of-the-art results on two widely adopted benchmarks for the traditional group activity recognition task (assuming individuals of the scene form a single group and predicting a single group activity label for the scene); iii) we introduce new annotations on an existing group activity dataset, re-purposing it for the social task.

READ FULL TEXT

page 2

page 9

page 10

research
07/12/2022

Hunting Group Clues with Transformers for Social Group Activity Recognition

This paper presents a novel framework for social group activity recognit...
research
06/16/2021

JRDB-Act: A Large-scale Multi-modal Dataset for Spatio-temporal Action, Social Group and Activity Detection

The availability of large-scale video action understanding datasets has ...
research
08/27/2012

Are You Imitating Me? Unsupervised Sparse Modeling for Group Activity Analysis from a Single Video

A framework for unsupervised group activity analysis from a single video...
research
06/03/2017

Learning Person Trajectory Representations for Team Activity Analysis

Activity analysis in which multiple people interact across a large space...
research
09/04/2017

A Nonparametric Model for Multimodal Collaborative Activities Summarization

Ego-centric data streams provide a unique opportunity to reason about jo...
research
03/04/2016

FLICA: A Framework for Leader Identification in Coordinated Activity

Leadership is an important aspect of social organization that affects th...
research
06/23/2015

SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

Studying free-standing conversational groups (FCGs) in unstructured soci...

Please sign up or login with your details

Forgot password? Click here to reset