Inferring User Facial Affect in Work-like Settings

Unlike the six basic emotions of happiness, sadness, fear, anger, disgust and surprise, modelling and predicting dimensional affect in terms of valence (positivity - negativity) and arousal (intensity) has proven to be more flexible, applicable and useful for naturalistic and real-world settings. In this paper, we aim to infer user facial affect when the user is engaged in multiple work-like tasks under varying difficulty levels (baseline, easy, hard and stressful conditions), including (i) an office-like setting where they undertake a task that is less physically demanding but requires greater mental strain; (ii) an assembly-line-like setting that requires the usage of fine motor skills; and (iii) an office-like setting representing teleworking and teleconferencing. In line with this aim, we first design a study with different conditions and gather multimodal data from 12 subjects. We then perform several experiments with various machine learning models and find that: (i) the display and prediction of facial affect vary from non-working to working settings; (ii) prediction capability can be boosted by using datasets captured in a work-like context; and (iii) segment-level (spectral representation) information is crucial in improving the facial affect prediction.

READ FULL TEXT

page 2

page 3

page 4

page 5

research
05/22/2019

Oculum afficit: Ocular Affect Recognition

Recognizing human affect and emotions is a problem that has a wide range...
research
06/16/2021

Toward Affective XAI: Facial Affect Analysis for Understanding Explainable Human-AI Interactions

As machine learning approaches are increasingly used to augment human de...
research
10/28/2020

Quantified Facial Temporal-Expressiveness Dynamics for Affect Analysis

The quantification of visual affect data (e.g. face images) is essential...
research
01/16/2020

Emotional Avatars: The Interplay between Affect and Ownership of a Virtual Body

Human bodies influence the owners' affect through posture, facial expres...
research
07/08/2021

Causal affect prediction model using a facial image sequence

Among human affective behavior research, facial expression recognition r...
research
08/23/2021

Modeling Dynamics of Facial Behavior for Mental Health Assessment

Facial action unit (FAU) intensities are popular descriptors for the ana...

Please sign up or login with your details

Forgot password? Click here to reset