The Brain-Inspired Decoder for Natural Visual Image Reconstruction

07/18/2022
by   Wenyi Li, et al.
0

Decoding images from brain activity has been a challenge. Owing to the development of deep learning, there are available tools to solve this problem. The decoded image, which aims to map neural spike trains to low-level visual features and high-level semantic information space. Recently, there are a few studies of decoding from spike trains, however, these studies pay less attention to the foundations of neuroscience and there are few studies that merged receptive field into visual image reconstruction. In this paper, we propose a deep learning neural network architecture with biological properties to reconstruct visual image from spike trains. As far as we know, we implemented a method that integrated receptive field property matrix into loss function at the first time. Our model is an end-to-end decoder from neural spike trains to images. We not only merged Gabor filter into auto-encoder which used to generate images but also proposed a loss function with receptive field properties. We evaluated our decoder on two datasets which contain macaque primary visual cortex neural spikes and salamander retina ganglion cells (RGCs) spikes. Our results show that our method can effectively combine receptive field features to reconstruct images, providing a new approach to visual reconstruction based on neural information.

READ FULL TEXT

page 2

page 3

page 4

page 5

research
04/30/2019

Reconstruction of Natural Visual Scenes from Neural Spikes with Deep Neural Networks

Neural coding is one of the central questions in systems neuroscience fo...
research
11/30/2021

Deep Auto-encoder with Neural Response

Artificial intelligence and neuroscience are deeply interactive. Artific...
research
01/04/2022

A Robust Visual Sampling Model Inspired by Receptive Field

Spike camera mimicking the retina fovea can report per-pixel luminance i...
research
08/09/2021

Using Deep Learning for Visual Decoding and Reconstruction from Brain Activity: A Review

This literature review will discuss the use of deep learning methods for...
research
06/16/2021

Over-and-Under Complete Convolutional RNN for MRI Reconstruction

Reconstructing magnetic resonance (MR) images from undersampled data is ...
research
06/02/2010

Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields

We introduce a new neural architecture and an unsupervised algorithm for...
research
06/01/2023

Second Sight: Using brain-optimized encoding models to align image distributions with human brain activity

Two recent developments have accelerated progress in image reconstructio...

Please sign up or login with your details

Forgot password? Click here to reset