Harmonizing the object recognition strategies of deep neural networks with humans

11/08/2022
by   Thomas Fel, et al.
0

The many successes of deep neural networks (DNNs) over the past decade have largely been driven by computational scale rather than insights from biological intelligence. Here, we explore if these trends have also carried concomitant improvements in explaining the visual strategies humans rely on for object recognition. We do this by comparing two related but distinct properties of visual strategies in humans and DNNs: where they believe important visual features are in images and how they use those features to categorize objects. Across 84 different DNNs trained on ImageNet and three independent datasets measuring the where and the how of human visual strategies for object recognition on those images, we find a systematic trade-off between DNN categorization accuracy and alignment with human visual strategies for object recognition. State-of-the-art DNNs are progressively becoming less aligned with humans as their accuracy improves. We rectify this growing issue with our neural harmonizer: a general-purpose training routine that both aligns DNN and human visual strategies and improves categorization accuracy. Our work represents the first demonstration that the scaling laws that are guiding the design of DNNs today have also produced worse models of human vision. We release our code and data at https://serre-lab.github.io/Harmonization to help the field build more human-like DNNs.

READ FULL TEXT

page 5

page 8

page 16

page 17

page 18

page 19

page 23

research
06/06/2023

Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex

One of the most impactful findings in computational neuroscience over th...
research
06/21/2017

Comparing deep neural networks against humans: object recognition when the signal gets weaker

Human visual object recognition is typically rapid and seemingly effortl...
research
06/05/2023

Adversarial alignment: Breaking the trade-off between the strength of an attack and its relevance to human perception

Deep neural networks (DNNs) are known to have a fundamental sensitivity ...
research
06/14/2021

Modeling Object Recognition in Newborn Chicks using Deep Neural Networks

In recent years, the brain and cognitive sciences have made great stride...
research
06/30/2020

Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency

A central problem in cognitive science and behavioural neuroscience as w...
research
06/20/2023

Computing a human-like reaction time metric from stable recurrent vision models

The meteoric rise in the adoption of deep neural networks as computation...
research
11/20/2016

Object Recognition with and without Objects

While recent deep neural networks have achieved a promising performance ...

Please sign up or login with your details

Forgot password? Click here to reset