Dissociable neural representations of adversarially perturbed images in deep neural networks and the human brain

by   Chi Zhang, et al.

Despite the remarkable similarities between deep neural networks (DNN) and the human brain as shown in previous studies, the fact that DNNs still fall behind humans in many visual tasks suggests that considerable differences still exist between the two systems. To probe their dissimilarities, we leverage adversarial noise (AN) and adversarial interference (AI) images that yield distinct recognition performance in a prototypical DNN (AlexNet) and human vision. The evoked activity by regular (RE) and adversarial images in both systems is thoroughly compared. We find that representational similarity between RE and adversarial images in the human brain resembles their perceptual similarity. However, such representation-perception association is disrupted in the DNN. Especially, the representational similarity between RE and AN images idiosyncratically increases from low- to high-level layers. Furthermore, forward encoding modeling reveals that the DNN-brain hierarchical correspondence proposed in previous studies only holds when the two systems process RE and AI images but not AN images. These results might be due to the deterministic modeling approach of current DNNs. Taken together, our results provide a complementary perspective on the comparison between DNNs and the human brain, and highlight the need to characterize their differences to further bridge artificial and human intelligence research.


page 36

page 37

page 39

page 40

page 41

page 42


Comparing deep neural networks against humans: object recognition when the signal gets weaker

Human visual object recognition is typically rapid and seemingly effortl...

A Complementary Learning Systems Approach to Temporal Difference Learning

Complementary Learning Systems (CLS) theory suggests that the brain uses...

A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions

Deep neural networks (DNNs) achieve excellent performance on standard cl...

Deep Neural Networks Abstract Like Humans

Deep neural networks (DNNs) have revolutionized AI due to their remarkab...

Understanding the Ability of Deep Neural Networks to Count Connected Components in Images

Humans can count very fast by subitizing, but slow substantially as the ...

BI AVAN: Brain inspired Adversarial Visual Attention Network

Visual attention is a fundamental mechanism in the human brain, and it i...

Do DNNs trained on Natural Images acquire Gestalt Properties?

Under some circumstances, humans tend to perceive individual elements as...

Please sign up or login with your details

Forgot password? Click here to reset