An encoding framework with brain inner state for natural image identification

08/22/2019
by   Hao Wu, et al.
4

Neural encoding and decoding, which aim to characterize the relationship between stimuli and brain activities, have emerged as an important area in cognitive neuroscience. Traditional encoding models, which focus on feature extraction and mapping, consider the brain as an input-output mapper without inner states. In this work, inspired by the fact that human brain acts like a state machine, we proposed a novel encoding framework that combines information from both the external world and the inner state to predict brain activity. The framework comprises two parts: forward encoding model that deals with visual stimuli and inner state model that captures influence from intrinsic connections in the brain. The forward model can be any traditional encoding model, making the framework flexible. The inner state model is a linear model to utilize information in the prediction residuals of the forward model. The proposed encoding framework can achieve much better performance on natural image identification from fMRI response than forwardonly models. The identification accuracy will decrease slightly with the dataset size increasing, but remain relatively stable with different identification methods. The results confirm that the new encoding framework is effective and robust when used for brain decoding.

READ FULL TEXT

page 1

page 7

research
11/26/2018

Mixture of Regression Experts in fMRI Encoding

fMRI semantic category understanding using linguistic encoding models at...
research
05/15/2021

NeuroGen: activation optimized image synthesis for discovery neuroscience

Functional MRI (fMRI) is a powerful technique that has allowed us to cha...
research
08/02/2023

Memory Encoding Model

We explore a new class of brain encoding model by adding memory-related ...
research
09/26/2019

ExpertoCoder: Capturing Divergent Brain Regions Using Mixture of Regression Experts

fMRI semantic category understanding using linguistic encoding models at...
research
10/05/2017

Anatomical Pattern Analysis for decoding visual stimuli in human brains

Background: A universal unanswered question in neuroscience and machine ...
research
07/26/2023

Retinotopy Inspired Brain Encoding Model and the All-for-One Training Recipe

Brain encoding models aim to predict brain voxel-wise responses to stimu...
research
02/21/2022

Same Cause; Different Effects in the Brain

To study information processing in the brain, neuroscientists manipulate...

Please sign up or login with your details

Forgot password? Click here to reset