BI AVAN: Brain inspired Adversarial Visual Attention Network

10/27/2022
by   Heng Huang, et al.
0

Visual attention is a fundamental mechanism in the human brain, and it inspires the design of attention mechanisms in deep neural networks. However, most of the visual attention studies adopted eye-tracking data rather than the direct measurement of brain activity to characterize human visual attention. In addition, the adversarial relationship between the attention-related objects and attention-neglected background in the human visual system was not fully exploited. To bridge these gaps, we propose a novel brain-inspired adversarial visual attention network (BI-AVAN) to characterize human visual attention directly from functional brain activity. Our BI-AVAN model imitates the biased competition process between attention-related/neglected objects to identify and locate the visual objects in a movie frame the human brain focuses on in an unsupervised manner. We use independent eye-tracking data as ground truth for validation and experimental results show that our model achieves robust and promising results when inferring meaningful human visual attention and mapping the relationship between brain activities and visual stimuli. Our BI-AVAN model contributes to the emerging field of leveraging the brain's functional architecture to inspire and guide the model design in artificial intelligence (AI), e.g., deep neural networks.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 9

page 10

research
10/01/2020

Neural encoding with visual attention

Visual perception is critically influenced by the focus of attention. Du...
research
02/13/2020

Recurrent Attention Model with Log-Polar Mapping is Robust against Adversarial Attacks

Convolutional neural networks are vulnerable to small ℓ^p adversarial at...
research
11/23/2018

Learning to attend in a brain-inspired deep neural network

Recent machine learning models have shown that including attention as a ...
research
05/03/2021

MemX: An Attention-Aware Smart Eyewear System for Personalized Moment Auto-capture

This work presents MemX: a biologically-inspired attention-aware eyewear...
research
12/22/2018

Dissociable neural representations of adversarially perturbed images in deep neural networks and the human brain

Despite the remarkable similarities between deep neural networks (DNN) a...
research
01/11/2022

Where Is My Mind (looking at)? Predicting Visual Attention from Brain Activity

Visual attention estimation is an active field of research at the crossr...
research
06/15/2022

Human Eyes Inspired Recurrent Neural Networks are More Robust Against Adversarial Noises

Compared to human vision, computer vision based on convolutional neural ...

Please sign up or login with your details

Forgot password? Click here to reset