Multimodal Depression Classification Using Articulatory Coordination Features And Hierarchical Attention Based Text Embeddings

02/13/2022

∙

Multimodal depression classification has gained immense popularity over the recent years. We develop a multimodal depression classification system using articulatory coordination features extracted from vocal tract variables and text transcriptions obtained from an automatic speech recognition tool that yields improvements of area under the receiver operating characteristics curve compared to uni-modal classifiers (7.5 respectively). We show that in the case of limited training data, a segment-level classifier can first be trained to then obtain a session-wise prediction without hindering the performance, using a multi-stage convolutional recurrent neural network. A text model is trained using a Hierarchical Attention Network (HAN). The multimodal system is developed by combining embeddings from the session-level audio model and the HAN text model

READ FULL TEXT

Multimodal Depression Classification Using Articulatory Coordination Features And Hierarchical Attention Based Text Embeddings

Speech based Depression Severity Level Classification Using a Multi-Stage Dilated CNN-LSTM Model

Replacing Human Audio with Synthetic Audio for On-device Unspoken Punctuation Prediction

Efficient Ensemble Architecture for Multimodal Acoustic and Textual Embeddings in Punctuation Restoration using Time-Delay Neural Networks

Multimodal Deep Learning for Mental Disorders Prediction from Audio Speech Samples

MultiQT: Multimodal Learning for Real-Time Question Tracking in Speech

Predicting Mood Disorder Symptoms with Remotely Collected Videos Using an Interpretable Multimodal Dynamic Attention Fusion Network

Deep Learning Based Generalized Models for Depression Classification

Multimodal Depression Classification Using Articulatory Coordination Features And Hierarchical Attention Based Text Embeddings

Related Research

Speech based Depression Severity Level Classification Using a Multi-Stage Dilated CNN-LSTM Model

Replacing Human Audio with Synthetic Audio for On-device Unspoken Punctuation Prediction

Efficient Ensemble Architecture for Multimodal Acoustic and Textual Embeddings in Punctuation Restoration using Time-Delay Neural Networks

Multimodal Deep Learning for Mental Disorders Prediction from Audio Speech Samples

MultiQT: Multimodal Learning for Real-Time Question Tracking in Speech

Predicting Mood Disorder Symptoms with Remotely Collected Videos Using an Interpretable Multimodal Dynamic Attention Fusion Network

Deep Learning Based Generalized Models for Depression Classification