Contributions of Shape, Texture, and Color in Visual Recognition

by   Yunhao Ge, et al.

We investigate the contributions of three important features of the human visual system (HVS)  –  shape, texture, and color   –  to object classification. We build a humanoid vision engine (HVE) that explicitly and separately computes shape, texture, and color features from images. The resulting feature vectors are then concatenated to support the final classification. We show that HVE can summarize and rank-order the contributions of the three features to object recognition. We use human experiments to confirm that both HVE and humans predominantly use some specific features to support the classification of specific classes (e.g., texture is the dominant feature to distinguish a zebra from other quadrupeds, both for humans and HVE). With the help of HVE, given any environment (dataset), we can summarize the most important features for the whole task (task-specific; e.g., color is the most important feature overall for classification with the CUB dataset), and for each class (class-specific; e.g., shape is the most important feature to recognize boats in the iLab-20M dataset). To demonstrate more usefulness of HVE, we use it to simulate the open-world zero-shot learning ability of humans with no attribute labeling. Finally, we show that HVE can also simulate human imagination ability with the combination of different features. We will open-source the HVE engine and corresponding datasets.


page 16

page 24

page 25

page 26

page 31

page 32

page 33

page 36


Texture features in medical image analysis: a survey

The texture is defined as spatial structure of the intensities of the pi...

What Do People See in a Twenty-Second Glimpse of Bivariate Vector Field Visualizations?

Little is known about how people learn from a brief glimpse of three-dim...

Role of Class-specific Features in Various Classification Frameworks for Human Epithelial (HEp-2) Cell Images

The antinuclear antibody detection with human epithelial cells is a popu...

Signal Strength and Noise Drive Feature Preference in CNN Image Classifiers

Feature preference in Convolutional Neural Network (CNN) image classifie...

Superpixel based Class-Semantic Texton Occurrences for Natural Roadside Vegetation Segmentation

Vegetation segmentation from roadside data is a field that has received ...

Neural Network Application on Foliage Plant Identification

Several researches in leaf identification did not include color informat...