Connecting metrics for shape-texture knowledge in computer vision

by   Tiago Oliveira, et al.

Modern artificial neural networks, including convolutional neural networks and vision transformers, have mastered several computer vision tasks, including object recognition. However, there are many significant differences between the behavior and robustness of these systems and of the human visual system. Deep neural networks remain brittle and susceptible to many changes in the image that do not cause humans to misclassify images. Part of this different behavior may be explained by the type of features humans and deep neural networks use in vision tasks. Humans tend to classify objects according to their shape while deep neural networks seem to rely mostly on texture. Exploring this question is relevant, since it may lead to better performing neural network architectures and to a better understanding of the workings of the vision system of primates. In this work, we advance the state of the art in our understanding of this phenomenon, by extending previous analyses to a much larger set of deep neural network architectures. We found that the performance of models in image classification tasks is highly correlated with their shape bias measured at the output and penultimate layer. Furthermore, our results showed that the number of neurons that represent shape and texture are strongly anti-correlated, thus providing evidence that there is competition between these two types of features. Finally, we observed that while in general there is a correlation between performance and shape bias, there are significant variations between architecture families.


page 1

page 2

page 3

page 4


Exploring the Origins and Prevalence of Texture Bias in Convolutional Neural Networks

Recent work has indicated that, unlike humans, ImageNet-trained CNNs ten...

Ikshana: A Theory of Human Scene Understanding Mechanism

In recent years, deep neural networks achieved state-of-the-art performa...

Degraded Polygons Raise Fundamental Questions of Neural Network Perception

It is well-known that modern computer vision systems often exhibit behav...

CVSNet: A Computer Implementation for Central Visual System of The Brain

In computer vision, different basic blocks are created around different ...

A Novel Neuron Model of Visual Processor

Simulating and imitating the neuronal network of humans or mammals is a ...

Leveraging Systematic Knowledge of 2D Transformations

The existing deep learning models suffer from out-of-distribution (o.o.d...

A Neural Algorithm of Artistic Style

In fine art, especially painting, humans have mastered the skill to crea...

Please sign up or login with your details

Forgot password? Click here to reset