A Neurorobotic Experiment for Crossmodal Conflict Resolution in Complex Environments

by   German I. Parisi, et al.

Crossmodal conflict resolution is a crucial component of robot sensorimotor coupling through interaction with the environment for swift and robust behaviour also in noisy conditions. In this paper, we propose a neurorobotic experiment in which an iCub robot exhibits human-like responses in a complex crossmodal environment. To better understand how humans deal with multisensory conflicts, we conducted a behavioural study exposing 33 subjects to congruent and incongruent dynamic audio-visual cues. In contrast to previous studies using simplified stimuli, we designed a scenario with four animated avatars and observed that the magnitude and extension of the visual bias are related to the semantics embedded in the scene, i.e., visual cues that are congruent with environmental statistics (moving lips and vocalization) induce a stronger bias. We propose a deep learning model that processes stereophonic sound, facial features, and body motion to trigger a discrete response resembling the collected behavioural data. After training, we exposed the iCub to the same experimental conditions as the human subjects, showing that the robot can replicate similar responses in real time. Our interdisciplinary work provides important insights into how crossmodal conflict resolution can be modelled in robots and introduces future research directions for the efficient combination of sensory drive with internally generated knowledge and expectations.


page 3

page 6


A trained humanoid robot can perform human-like crossmodal social attention conflict resolution

Due to the COVID-19 pandemic, robots could be seen as potential resource...

Assessing the Contribution of Semantic Congruency to Multisensory Integration and Conflict Resolution

The efficient integration of multisensory observations is a key property...

Using Variable Natural Environment Brain-Computer Interface Stimuli for Real-time Humanoid Robot Navigation

This paper addresses the challenge of humanoid robot teleoperation in a ...

Conflict Detection and Resolution in Table Top Scenarios for Human-Robot Interaction

As in any interaction process, misunderstandings, ambiguity, and failure...

iCub Being Social: Exploiting Social Cues for Interactive Object Detection Learning

Performing joint interaction requires constant mutual monitoring of own ...

Robots as Actors in a Film: No War, A Robot Story

Will the Third World War be fought by robots? This short film is a light...

Training development for multisensory data analysis

Perception is a process that requires a great deal of mental processing,...

Please sign up or login with your details

Forgot password? Click here to reset