Personalized visual encoding model construction with small data

02/04/2022
by   Zijin Gu, et al.
2

Encoding models that predict brain response patterns to stimuli are one way to capture this relationship between variability in bottom-up neural systems and individual's behavior or pathological state. However, they generally need a large amount of training data to achieve optimal accuracy. Here, we propose and test an alternative personalized ensemble encoding model approach to utilize existing encoding models, to create encoding models for novel individuals with relatively little stimuli-response data. We show that these personalized ensemble encoding models trained with small amounts of data for a specific individual, i.e.  400 image-response pairs, achieve accuracy not different from models trained on  24,000 image-response pairs for the same individual. Importantly, the personalized ensemble encoding models preserve patterns of inter-individual variability in the image-response relationship. Additionally, we use our personalized ensemble encoding model within the recently developed NeuroGen framework to generate optimal stimuli designed to maximize specific regions' activations for a specific individual. We show that the inter-individual differences in face area responses to images of dog vs human faces observed previously is replicated using NeuroGen with the ensemble encoding model. Finally, and most importantly, we show the proposed approach is robust against domain shift by validating on a prospectively collected set of image-response data in novel individuals with a different scanner and experimental setup. Our approach shows the potential to use previously collected, deeply sampled data to efficiently create accurate, personalized encoding models and, subsequently, personalized optimal synthetic images for new individuals scanned under different experimental conditions.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 7

page 8

page 10

research
05/15/2021

NeuroGen: activation optimized image synthesis for discovery neuroscience

Functional MRI (fMRI) is a powerful technique that has allowed us to cha...
research
06/29/2020

A shared neural encoding model for the prediction of subject-specific fMRI response

The increasing popularity of naturalistic paradigms in fMRI (such as mov...
research
07/20/2020

Bayesian optimization for automatic design of face stimuli

Investigating the cognitive and neural mechanisms involved with face pro...
research
06/16/2023

Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses

Neuroprostheses show potential in restoring lost sensory function and en...
research
08/02/2023

Memory Encoding Model

We explore a new class of brain encoding model by adding memory-related ...
research
04/19/2023

Personalized State Anxiety Detection: An Empirical Study with Linguistic Biomarkers and A Machine Learning Pipeline

Individuals high in social anxiety symptoms often exhibit elevated state...
research
04/30/2021

Simple and Cheap Setup for Timing Tapping Responses Synchronized to Auditory Stimuli

Measuring human capabilities to synchronize in time, adapt to perturbati...

Please sign up or login with your details

Forgot password? Click here to reset