SwiFT: Swin 4D fMRI Transformer

by   Peter Yongho Kim, et al.

The modeling of spatiotemporal brain dynamics from high-dimensional data, such as 4D functional MRI, is a formidable task in neuroscience. To address this challenge, we present SwiFT (Swin 4D fMRI Transformer), a Swin Transformer architecture that can learn brain dynamics directly from 4D functional brain MRI data in a memory and computation-efficient manner. SwiFT achieves this by implementing a 4D window multi-head self-attention mechanism and absolute positional embeddings. We evaluate SwiFT using multiple largest-scale human functional brain imaging datasets in tasks such as predicting sex, age, and cognitive intelligence. Our experimental outcomes reveal that SwiFT consistently outperforms recent state-of-the-art models. To the best of our knowledge, SwiFT is the first Swin Transformer architecture that can process dimensional spatiotemporal brain functional data in an end-to-end fashion. Furthermore, due to the end-to-end learning capability, we also show that contrastive loss-based self-supervised pre-training of SwiFT is also feasible for achieving improved performance on a downstream task. We believe that our work holds substantial potential in facilitating scalable learning of functional brain imaging in neuroscience research by reducing the hurdles associated with applying Transformer models to high-dimensional fMRI.


page 4

page 9


Effective and Interpretable fMRI Analysis via Functional Brain Network Generation

Recent studies in neuroscience show great potential of functional brain ...

DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI

Deep learning analyses have offered sensitivity leaps in detection of co...

Pre-training and Fine-tuning Transformers for fMRI Prediction Tasks

We present the TFF Transformer framework for the analysis of functional ...

Generative artificial intelligence-enabled dynamic detection of nicotine-related circuits

The identification of addiction-related circuits is critical for explain...

BolT: Fused Window Transformers for fMRI Time Series Analysis

Functional magnetic resonance imaging (fMRI) enables examination of inte...

Deep learning for neuroimaging: a validation study

Deep learning methods have recently made notable advances in the tasks o...

Learning Neural Representations of Human Cognition across Many fMRI Studies

Cognitive neuroscience is enjoying rapid increase in extensive public br...

Please sign up or login with your details

Forgot password? Click here to reset