EEG to fMRI Synthesis: Is Deep Learning a candidate?

by   David Calhas, et al.

Advances on signal, image and video generation underly major breakthroughs on generative medical imaging tasks, including Brain Image Synthesis. Still, the extent to which functional Magnetic Ressonance Imaging (fMRI) can be mapped from the brain electrophysiology remains largely unexplored. This work provides the first comprehensive view on how to use state-of-the-art principles from Neural Processing to synthesize fMRI data from electroencephalographic (EEG) data. Given the distinct spatiotemporal nature of haemodynamic and electrophysiological signals, this problem is formulated as the task of learning a mapping function between multivariate time series with highly dissimilar structures. A comparison of state-of-the-art synthesis approaches, including Autoencoders, Generative Adversarial Networks and Pairwise Learning, is undertaken. Results highlight the feasibility of EEG to fMRI brain image mappings, pinpointing the role of current advances in Machine Learning and showing the relevance of upcoming contributions to further improve performance. EEG to fMRI synthesis offers a way to enhance and augment brain image data, and guarantee access to more affordable, portable and long-lasting protocols of brain activity monitoring. The code used in this manuscript is available in Github and the datasets are open source.



There are no comments yet.


page 1

page 7


A comparison of single-trial EEG classification and EEG-informed fMRI across three MR compatible EEG recording systems

Simultaneously recorded electroencephalography (EEG) and functional magn...

EEG-GAN: Generative adversarial networks for electroencephalograhic (EEG) brain signals

Generative adversarial networks (GANs) are recently highly successful in...

Augmenting interictal mapping with neurovascular coupling biomarkers by structured factorization of epileptic EEG and fMRI data

EEG-correlated fMRI analysis is widely used to detect regional blood oxy...

FMRI data augmentation via synthesis

We present an empirical evaluation of fMRI data augmentation via synthes...

Latent neural source recovery via transcoding of simultaneous EEG-fMRI

Simultaneous EEG-fMRI is a multi-modal neuroimaging technique that provi...

Product semantics translation from brain activity via adversarial learning

A small change of design semantics may affect a user's satisfaction with...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.