Looking At The Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress

by   Weizhe Lin, et al.

Psychological distress is a significant and growing issue in society. Automatic detection, assessment, and analysis of such distress is an active area of research. Compared to modalities such as face, head, and vocal, research investigating the use of the body modality for these tasks is relatively sparse. This is, in part, due to the limited available datasets and difficulty in automatically extracting useful body features. Recent advances in pose estimation and deep learning have enabled new approaches to this modality and domain. To enable this research, we have collected and analyzed a new dataset containing full body videos for short interviews and self-reported distress labels. We propose a novel method to automatically detect self-adaptors and fidgeting, a subset of self-adaptors that has been shown to be correlated with psychological distress. We perform analysis on statistical body gestures and fidgeting features to explore how distress levels affect participants' behaviors. We then propose a multi-modal approach that combines different feature representations using Multi-modal Deep Denoising Auto-Encoders and Improved Fisher Vector Encoding. We demonstrate that our proposed model, combining audio-visual features with automatically detected fidgeting behavioral cues, can successfully predict distress levels in a dataset labeled with self-reported anxiety and depression levels.


page 10

page 13


Vision based body gesture meta features for Affective Computing

Early detection of psychological distress is key to effective treatment....

Flexible-modal Deception Detection with Audio-Visual Adapter

Detecting deception by human behaviors is vital in many fields such as c...

Survey on Emotional Body Gesture Recognition

Automatic emotion recognition has become a trending research topic in th...

Anticipatory Detection of Compulsive Body-focused Repetitive Behaviors with Wearables

Body-focused repetitive behaviors (BFRBs), like face-touching or skin-pi...

BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis

Achieving realistic, vivid, and human-like synthesized conversational ge...

AIMusicGuru: Music Assisted Human Pose Correction

Pose Estimation techniques rely on visual cues available through observa...

In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains

Medical applications have benefited from the rapid advancement in comput...

Please sign up or login with your details

Forgot password? Click here to reset