Modality Fusion Network and Personalized Attention in Momentary Stress Detection in the Wild

07/19/2021
by   Han Yu, et al.
5

Multimodal wearable physiological data in daily life have been used to estimate self-reported stress labels. However, missing data modalities in data collection makes it challenging to leverage all the collected samples. Besides, heterogeneous sensor data and labels among individuals add challenges in building robust stress detection models. In this paper, we proposed a modality fusion network (MFN) to train models and infer self-reported binary stress labels under both complete and incomplete modality conditions. In addition, we applied personalized attention (PA) strategy to leverage personalized representation along with the generalized one-size-fits-all model. We evaluated our methods on a multimodal wearable sensor dataset (N=41) including galvanic skin response (GSR) and electrocardiogram (ECG). Compared to the baseline method using the samples with complete modalities, the performance of the MFN improved by 1.6 showed a 2.3 reduction in personalized model parameter size (9.1 MB) compared to the previous state-of-the-art transfer learning strategy (29.3 MB).

READ FULL TEXT

page 1

page 4

page 6

page 7

research
07/07/2023

Personalized Prediction of Recurrent Stress Events Using Self-Supervised Learning on Multimodal Time-Series Data

Chronic stress can significantly affect physical and mental health. The ...
research
02/22/2022

Semi-Supervised Learning and Data Augmentation in Wearable-based Momentary Stress Detection in the Wild

Physiological and behavioral data collected from wearable or mobile sens...
research
06/26/2019

Personalized Student Stress Prediction with Deep Multitask Network

With the growing popularity of wearable devices, the ability to utilize ...
research
03/07/2023

ERUDITE: Human-in-the-Loop IoT for an Adaptive Personalized Learning System

Thanks to the rapid growth in wearable technologies and recent advanceme...
research
08/02/2021

Data Collection and Labeling of Real-Time IoT-Enabled Bio-Signals in Everyday Settings for Mental Health Improvement

Real-time physiological data collection and analysis play a central role...
research
07/17/2022

Fusion of Physiological and Behavioural Signals on SPD Manifolds with Application to Stress and Pain Detection

Existing multimodal stress/pain recognition approaches generally extract...
research
06/15/2023

Employing Multimodal Machine Learning for Stress Detection

In the current age, human lifestyle has become more knowledge oriented l...

Please sign up or login with your details

Forgot password? Click here to reset