Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detection

08/17/2021
by   Leena Mathur, et al.
0

Automated systems that detect the social behavior of deception can enhance human well-being across medical, social work, and legal domains. Labeled datasets to train supervised deception detection models can rarely be collected for real-world, high-stakes contexts. To address this challenge, we propose the first unsupervised approach for detecting real-world, high-stakes deception in videos without requiring labels. This paper presents our novel approach for affect-aware unsupervised Deep Belief Networks (DBN) to learn discriminative representations of deceptive and truthful behavior. Drawing on psychology theories that link affect and deception, we experimented with unimodal and multimodal DBN-based approaches trained on facial valence, facial arousal, audio, and visual features. In addition to using facial affect as a feature on which DBN models are trained, we also introduce a DBN training procedure that uses facial affect as an aligner of audio-visual representations. We conducted classification experiments with unsupervised Gaussian Mixture Model clustering to evaluate our approaches. Our best unsupervised approach (trained on facial valence and visual features) achieved an AUC of 80 ability and performing comparably to fully-supervised models. Our results motivate future work on unsupervised, affect-aware computational approaches for detecting deception and other social behaviors in the wild.

READ FULL TEXT
research
08/31/2020

Introducing Representations of Facial Affect in Automated Multimodal Deception Detection

Automated deception detection systems can enhance health, justice, and s...
research
02/06/2021

Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection

Automated systems that detect deception in high-stakes situations can en...
research
07/31/2022

Towards Intercultural Affect Recognition: Audio-Visual Affect Recognition in the Wild Across Six Cultures

In our multicultural world, affect-aware AI systems that support humans ...
research
09/07/2022

Evaluating Temporal Patterns in Applied Infant Affect Recognition

Agents must monitor their partners' affective states continuously in ord...
research
10/28/2020

Quantified Facial Temporal-Expressiveness Dynamics for Affect Analysis

The quantification of visual affect data (e.g. face images) is essential...
research
11/25/2019

Detecting Unknown Behaviors by Pre-defined Behaviours: An Bayesian Non-parametric Approach

An automatic mouse behavior recognition system can considerably reduce t...
research
05/18/2023

From the Lab to the Wild: Affect Modeling via Privileged Information

How can we reliably transfer affect models trained in controlled laborat...

Please sign up or login with your details

Forgot password? Click here to reset