Attend and Decode: 4D fMRI Task State Decoding Using Attention Models

04/10/2020
by   Sam Nguyen, et al.
9

Functional magnetic resonance imaging (fMRI) is a neuroimaging modality that captures the blood oxygen level in a subject's brain while the subject performs a variety of functional tasks under different conditions. Given fMRI data, the problem of inferring the task, known as task state decoding, is challenging due to the high dimensionality (hundreds of million sampling points per datum) and complex spatio-temporal blood flow patterns inherent in the data. In this work, we propose to tackle the fMRI task state decoding problem by casting it as a 4D spatio-temporal classification problem. We present a novel architecture called Brain Attend and Decode (BAnD), that uses residual convolutional neural networks for spatial feature extraction and self-attention mechanisms for temporal modeling. We achieve significant performance gain compared to previous works on a 7-task benchmark from the large-scale Human Connectome Project (HCP) dataset. We also investigate the transferability of BAnD's extracted features on unseen HCP tasks, either by freezing the spatial feature extraction layers and retraining the temporal model, or finetuning the entire model. The pre-trained features from BAnD are useful on similar tasks while finetuning them yields competitive results on unseen tasks/conditions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/28/2021

Understanding Cognitive Fatigue from fMRI Scans with Self-supervised Learning

Functional magnetic resonance imaging (fMRI) is a neuroimaging technique...
research
04/21/2020

4D Spatio-Temporal Deep Learning with 4D fMRI Data for Autism Spectrum Disorder Classification

Autism spectrum disorder (ASD) is associated with behavioral and communi...
research
02/14/2020

A Hybrid 3DCNN and 3DC-LSTM based model for 4D Spatio-temporal fMRI data: An ABIDE Autism Classification study

Functional Magnetic Resonance Imaging (fMRI) captures the temporal dynam...
research
12/23/2014

Learning Deep Temporal Representations for Brain Decoding

Functional magnetic resonance imaging produces high dimensional data, wi...
research
07/02/2020

Deep brain state classification of MEG data

Neuroimaging techniques have shown to be useful when studying the brain'...
research
04/20/2022

Disentangling Spatial-Temporal Functional Brain Networks via Twin-Transformers

How to identify and characterize functional brain networks (BN) is funda...
research
06/13/2018

fMRI Semantic Category Decoding using Linguistic Encoding of Word Embeddings

The dispute of how the human brain represents conceptual knowledge has b...

Please sign up or login with your details

Forgot password? Click here to reset