Deep Auto-encoder with Neural Response

11/30/2021
by   Xuming Ran, et al.
32

Artificial intelligence and neuroscience are deeply interactive. Artificial neural networks (ANNs) have been a versatile tool to study the neural representation in the ventral visual stream, and the knowledge in neuroscience in return inspires ANN models to improve performance in the task. However, how to merge these two directions into a unified model has less studied. Here, we propose a hybrid model, called deep auto-encoder with the neural response (DAE-NR), which incorporates the information from the visual cortex into ANNs to achieve better image reconstruction and higher neural representation similarity between biological and artificial neurons. Specifically, the same visual stimuli (i.e., natural images) are input to both the mice brain and DAE-NR. The DAE-NR jointly learns to map a specific layer of the encoder network to the biological neural responses in the ventral visual stream by a mapping function and to reconstruct the visual input by the decoder. Our experiments demonstrate that if and only if with the joint learning, DAE-NRs can (i) improve the performance of image reconstruction and (ii) increase the representational similarity between biological neurons and artificial neurons. The DAE-NR offers a new perspective on the integration of computer vision and visual neuroscience.

READ FULL TEXT

page 2

page 3

page 4

page 9

page 10

research
09/22/2021

Training Deep Spiking Auto-encoders without Bursting or Dying Neurons through Regularization

Spiking neural networks are a promising approach towards next-generation...
research
07/18/2022

The Brain-Inspired Decoder for Natural Visual Image Reconstruction

Decoding images from brain activity has been a challenge. Owing to the d...
research
06/08/2021

Object Based Attention Through Internal Gating

Object-based attention is a key component of the visual system, relevant...
research
10/22/2020

Factorized Neural Processes for Neural Processes: K-Shot Prediction of Neural Responses

In recent years, artificial neural networks have achieved state-of-the-a...
research
06/17/2022

The Sensorium competition on predicting large-scale mouse primary visual cortex activity

The neural underpinning of the biological visual system is challenging t...
research
12/09/2020

Binding and Perspective Taking as Inference in a Generative Neural Network Model

The ability to flexibly bind features into coherent wholes from differen...

Please sign up or login with your details

Forgot password? Click here to reset