Cogni-Net: Cognitive Feature Learning through Deep Visual Perception

11/01/2018
by   Pranay Mukherjee, et al.
10

Can we ask computers to recognize what we see from brain signals alone? Our paper seeks to utilize the knowledge learnt in the visual domain by popular pre-trained vision models and use it to teach a recurrent model being trained on brain signals to learn a discriminative manifold of the human brain's cognition of different visual object categories in response to perceived visual cues. For this we make use of brain EEG signals triggered from visual stimuli like images and leverage the natural synchronization between images and their corresponding brain signals to learn a novel representation of the cognitive feature space. The concept of knowledge distillation has been used here for training the deep cognition model, CogniNet[The source code of the proposed system is publicly available at https://www.github.com/53X/CogniNET], by employing a student-teacher learning technique in order to bridge the process of inter-modal knowledge transfer. The proposed novel architecture obtains state-of-the-art results, significantly surpassing other existing models. The experiments performed by us also suggest that if visual stimuli information like brain EEG signals can be gathered on a large scale, then that would help to obtain a better understanding of the largely unexplored domain of human brain cognition.

READ FULL TEXT

page 2

page 4

research
09/01/2016

Deep Learning Human Mind for Automated Visual Classification

What if we could effectively read the mind and transfer human visual cap...
research
07/27/2023

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

Seeing is believing, however, the underlying mechanism of how human visu...
research
09/03/2018

Deep Learning of Human Perception in Audio Event Classification

In this paper, we introduce our recent studies on human perception in au...
research
10/13/2022

Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features

Decoding human visual neural representations is a challenging task with ...
research
10/25/2018

Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Features

This paper tackles the problem of learning brain-visual representations ...
research
11/25/2018

Nonlinear Dynamics of Binocular Rivalry: A Comparative Study

When our eyes are presented with the same image, the brain processes it ...
research
07/26/2023

Retinotopy Inspired Brain Encoding Model and the All-for-One Training Recipe

Brain encoding models aim to predict brain voxel-wise responses to stimu...

Please sign up or login with your details

Forgot password? Click here to reset