A Network-based Multimodal Data Fusion Approach for Characterizing Dynamic Multimodal Physiological Patterns

by   Miaolin Fan, et al.
Northeastern University

Characterizing the dynamic interactive patterns of complex systems helps gain in-depth understanding of how components interrelate with each other while performing certain functions as a whole. In this study, we present a novel multimodal data fusion approach to construct a complex network, which models the interactions of biological subsystems in the human body under emotional states through physiological responses. Joint recurrence plot and temporal network metrics are employed to integrate the multimodal information at the signal level. A benchmark public dataset of is used for evaluating our model.


Low Rank Fusion based Transformers for Multimodal Sequences

Our senses individually work in a coordinated fashion to express our emo...

Global modeling of transcriptional responses in interaction networks

Motivation: Cell-biological processes are regulated through a complex ne...

A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances

Affective computing plays a key role in human-computer interactions, ent...

Facial Expression and Peripheral Physiology Fusion to Decode Individualized Affective Experience

In this paper, we present a multimodal approach to simultaneously analyz...

The dynamics of the stomatognathic system from 4D multimodal data

The purpose of this chapter is to discuss methods of acquisition, visual...

A novel multimodal fusion network based on a joint coding model for lane line segmentation

There has recently been growing interest in utilizing multimodal sensors...

Machine learning approaches to understand the influence of urban environments on human's physiological response

This research proposes a framework for signal processing and information...

Please sign up or login with your details

Forgot password? Click here to reset