Generalisation in humans and deep neural networks

08/27/2018
by   Robert Geirhos, et al.
4

We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations. First, using three well known DNNs (ResNet-152, VGG-19, GoogLeNet) we find the human visual system to be more robust to nearly all of the tested image manipulations, and we observe progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker. Secondly, we show that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on, yet they display extremely poor generalisation abilities when tested on other distortion types. For example, training on salt-and-pepper noise does not imply robustness on uniform white noise and vice versa. Thus, changes in the noise distribution between training and testing constitutes a crucial challenge to deep learning vision systems that can be systematically addressed in a lifelong machine learning approach. Our new dataset consisting of 83K carefully measured human psychophysical trials provide a useful reference for lifelong robustness against image degradations set by the human visual system.

READ FULL TEXT

page 4

page 7

page 21

page 22

page 23

page 24

page 25

page 26

research
06/21/2017

Comparing deep neural networks against humans: object recognition when the signal gets weaker

Human visual object recognition is typically rapid and seemingly effortl...
research
05/06/2017

A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions

Deep neural networks (DNNs) achieve excellent performance on standard cl...
research
05/20/2022

The developmental trajectory of object recognition robustness: children are like small adults but unlike big deep neural networks

In laboratory object recognition tasks based on undistorted photographs,...
research
06/06/2023

Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex

One of the most impactful findings in computational neuroscience over th...
research
05/07/2019

Representation of White- and Black-Box Adversarial Examples in Deep Neural Networks and Humans: A Functional Magnetic Resonance Imaging Study

The recent success of brain-inspired deep neural networks (DNNs) in solv...
research
08/29/2023

Classification robustness to common optical aberrations

Computer vision using deep neural networks (DNNs) has brought about semi...
research
02/08/2019

Minimal Images in Deep Neural Networks: Fragile Object Recognition in Natural Images

The human ability to recognize objects is impaired when the object is no...

Please sign up or login with your details

Forgot password? Click here to reset