Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation

07/26/2022
by   Michal Balazia, et al.
5

Body language is an eye-catching social signal and its automatic analysis can significantly advance artificial intelligence systems to understand and actively participate in social interactions. While computer vision has made impressive progress in low-level tasks like head and body pose estimation, the detection of more subtle behaviors such as gesturing, grooming, or fumbling is not well explored. In this paper we present BBSI, the first set of annotations of complex Bodily Behaviors embedded in continuous Social Interactions in a group setting. Based on previous work in psychology, we manually annotated 26 hours of spontaneous human behavior in the MPIIGroupInteraction dataset with 15 distinct body language classes. We present comprehensive descriptive statistics on the resulting dataset as well as results of annotation quality evaluations. For automatic detection of these behaviors, we adapt the Pyramid Dilated Attention Network (PDAN), a state-of-the-art approach for human action detection. We perform experiments using four variants of spatial-temporal features as input to PDAN: Two-Stream Inflated 3D CNN, Temporal Segment Networks, Temporal Shift Module and Swin Transformer. Results are promising and indicate a great room for improvement in this difficult task. Representing a key piece in the puzzle towards automatic understanding of social behavior, BBSI is fully available to the research community.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 12

page 13

page 14

research
09/19/2023

MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings

Bodily behavioral language is an important social cue, and its automated...
research
04/22/2020

Recursive Social Behavior Graph for Trajectory Prediction

Social interaction is an important topic in human trajectory prediction ...
research
06/02/2023

Backchannel Detection and Agreement Estimation from Video with Transformer Networks

Listeners use short interjections, so-called backchannels, to signify at...
research
05/06/2019

Emergent Leadership Detection Across Datasets

Automatic detection of emergent leaders in small groups from nonverbal b...
research
06/23/2015

SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

Studying free-standing conversational groups (FCGs) in unstructured soci...
research
08/08/2022

Social Simulacra: Creating Populated Prototypes for Social Computing Systems

Social computing prototypes probe the social behaviors that may arise in...
research
12/09/2016

Panoptic Studio: A Massively Multiview System for Social Interaction Capture

We present an approach to capture the 3D motion of a group of people eng...

Please sign up or login with your details

Forgot password? Click here to reset