Contributions of Shape, Texture, and Color in Visual Recognition

07/19/2022
by   Yunhao Ge, et al.
60

We investigate the contributions of three important features of the human visual system (HVS)  –  shape, texture, and color   –  to object classification. We build a humanoid vision engine (HVE) that explicitly and separately computes shape, texture, and color features from images. The resulting feature vectors are then concatenated to support the final classification. We show that HVE can summarize and rank-order the contributions of the three features to object recognition. We use human experiments to confirm that both HVE and humans predominantly use some specific features to support the classification of specific classes (e.g., texture is the dominant feature to distinguish a zebra from other quadrupeds, both for humans and HVE). With the help of HVE, given any environment (dataset), we can summarize the most important features for the whole task (task-specific; e.g., color is the most important feature overall for classification with the CUB dataset), and for each class (class-specific; e.g., shape is the most important feature to recognize boats in the iLab-20M dataset). To demonstrate more usefulness of HVE, we use it to simulate the open-world zero-shot learning ability of humans with no attribute labeling. Finally, we show that HVE can also simulate human imagination ability with the combination of different features. We will open-source the HVE engine and corresponding datasets.

READ FULL TEXT

page 16

page 24

page 25

page 26

page 31

page 32

page 33

page 36

08/02/2022

Texture features in medical image analysis: a survey

The texture is defined as spatial structure of the intensities of the pi...
05/07/2019

What Do People See in a Twenty-Second Glimpse of Bivariate Vector Field Visualizations?

Little is known about how people learn from a brief glimpse of three-dim...
10/30/2018

Role of Class-specific Features in Various Classification Frameworks for Human Epithelial (HEp-2) Cell Images

The antinuclear antibody detection with human epithelial cells is a popu...
01/19/2022

Signal Strength and Noise Drive Feature Preference in CNN Image Classifiers

Feature preference in Convolutional Neural Network (CNN) image classifie...
02/24/2018

Superpixel based Class-Semantic Texton Occurrences for Natural Roadside Vegetation Segmentation

Vegetation segmentation from roadside data is a field that has received ...
11/20/2013

Neural Network Application on Foliage Plant Identification

Several researches in leaf identification did not include color informat...