Assessing the Contribution of Semantic Congruency to Multisensory Integration and Conflict Resolution

10/15/2018
by   Di Fu, et al.
0

The efficient integration of multisensory observations is a key property of the brain that yields the robust interaction with the environment. However, artificial multisensory perception remains an open issue especially in situations of sensory uncertainty and conflicts. In this work, we extend previous studies on audio-visual (AV) conflict resolution in complex environments. In particular, we focus on quantitatively assessing the contribution of semantic congruency during an AV spatial localization task. In addition to conflicts in the spatial domain (i.e. spatially misaligned stimuli), we consider gender-specific conflicts with male and female avatars. Our results suggest that while semantically related stimuli affect the magnitude of the visual bias (perceptually shifting the location of the sound towards a semantically congruent visual cue), humans still strongly rely on environmental statistics to solve AV conflicts. Together with previously reported results, this work contributes to a better understanding of how multisensory integration and conflict resolution can be modelled in artificial agents and robots operating in real-world environments.

READ FULL TEXT
research
02/28/2018

A Neurorobotic Experiment for Crossmodal Conflict Resolution in Complex Environments

Crossmodal conflict resolution is a crucial component of robot sensorimo...
research
12/02/2019

Optimality and limitations of audio-visual integration for cognitive systems

Multimodal integration is an important process in perceptual decision-ma...
research
11/02/2021

A trained humanoid robot can perform human-like crossmodal social attention conflict resolution

Due to the COVID-19 pandemic, robots could be seen as potential resource...
research
07/13/2018

Towards Modeling the Interaction of Spatial-Associative Neural Network Representations for Multisensory Perception

Our daily perceptual experience is driven by different neural mechanisms...
research
03/14/2020

Audio-Visual Spatial Aligment Requirements of Central and Peripheral Object Events

Immersive audio-visual perception relies on the spatial integration of b...
research
11/29/2011

Developing Embodied Multisensory Dialogue Agents

A few decades of work in the AI field have focused efforts on developing...

Please sign up or login with your details

Forgot password? Click here to reset