Does the Goal Matter? Emotion Recognition Tasks Can Change the Social Value of Facial Mimicry towards Artificial Agents

by   Giulia Perugia, et al.

In this paper, we present a study aimed at understanding whether the embodiment and humanlikeness of an artificial agent can affect people's spontaneous and instructed mimicry of its facial expressions. The study followed a mixed experimental design and revolved around an emotion recognition task. Participants were randomly assigned to one level of humanlikeness (between-subject variable: humanlike, characterlike, or morph facial texture of the artificial agents) and observed the facial expressions displayed by a human (control) and three artificial agents differing in embodiment (within-subject variable: video-recorded robot, physical robot, and virtual agent). To study both spontaneous and instructed facial mimicry, we divided the experimental sessions into two phases. In the first phase, we asked participants to observe and recognize the emotions displayed by the agents. In the second phase, we asked them to look at the agents' facial expressions, replicate their dynamics as closely as possible, and then identify the observed emotions. In both cases, we assessed participants' facial expressions with an automated Action Unit (AU) intensity detector. Contrary to our hypotheses, our results disclose that the agent that was perceived as the least uncanny, and most anthropomorphic, likable, and co-present, was the one spontaneously mimicked the least. Moreover, they show that instructed facial mimicry negatively predicts spontaneous facial mimicry. Further exploratory analyses revealed that spontaneous facial mimicry appeared when participants were less certain of the emotion they recognized. Hence, we postulate that an emotion recognition goal can flip the social value of facial mimicry as it transforms a likable artificial agent into a distractor.


page 7

page 8

page 9

page 11


Do you see what I see? Taking perspective of others using facial images

Albeit many HCI / emotion recognition studies use facial expressive imag...

Multi-Cue Adaptive Emotion Recognition Network

Expressing and identifying emotions through facial and physical expressi...

A comparative study of emotion recognition methods using facial expressions

Understanding the facial expressions of our interlocutor is important to...

Dynamic Facial Expression of Emotion Made Easy

Facial emotion expression for virtual characters is used in a wide varie...

Sparsity in Dynamics of Spontaneous Subtle Emotions: Analysis & Application

Spontaneous subtle emotions are expressed through micro-expressions, whi...

An Ensemble with Shared Representations Based on Convolutional Networks for Continually Learning Facial Expressions

Social robots able to continually learn facial expressions could progres...

Emotion recognition using a glasses-type wearable device via multi-channel facial responses

We present a glasses type wearable device to detect emotions from a huma...

Please sign up or login with your details

Forgot password? Click here to reset