DeepAI
Log In Sign Up

Multi-modal fusion with gating using audio, lexical and disfluency features for Alzheimer's Dementia recognition from spontaneous speech

06/17/2021
by   Morteza Rohanian, et al.
0

This paper is a submission to the Alzheimer's Dementia Recognition through Spontaneous Speech (ADReSS) challenge, which aims to develop methods that can assist in the automated prediction of severity of Alzheimer's Disease from speech data. We focus on acoustic and natural language features for cognitive impairment detection in spontaneous speech in the context of Alzheimer's Disease Diagnosis and the mini-mental state examination (MMSE) score prediction. We proposed a model that obtains unimodal decisions from different LSTMs, one for each modality of text and audio, and then combines them using a gating mechanism for the final prediction. We focused on sequential modelling of text and audio and investigated whether the disfluencies present in individuals' speech relate to the extent of their cognitive impairment. Our results show that the proposed classification and regression schemes obtain very promising results on both development and test sets. This suggests Alzheimer's Disease can be detected successfully with sequence modeling of the speech data of medical sessions.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/11/2020

Alzheimer's Dementia Detection from Audio and Text Modalities

Automatic detection of Alzheimer's dementia by speech processing is enha...
04/14/2020

Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challenge

The ADReSS Challenge at INTERSPEECH 2020 defines a shared task through w...
11/30/2020

Multi-Modal Detection of Alzheimer's Disease from Speech and Text

Reliable detection of the prodromal stages of Alzheimer's disease (AD) r...
10/20/2017

Spoken Language Biomarkers for Detecting Cognitive Impairment

In this study we developed an automated system that evaluates speech and...
07/13/2020

Stutter Diagnosis and Therapy System Based on Deep Learning

Stuttering, also called stammering, is a communication disorder that bre...
09/02/2020

Detecting Parkinson's Disease from Speech-task in an accessible and interpretable manner

Every nine minutes a person is diagnosed with Parkinson's Disease (PD) i...
06/02/2022

Detecting the Severity of Major Depressive Disorder from Speech: A Novel HARD-Training Methodology

Major Depressive Disorder (MDD) is a common worldwide mental health issu...