TransHP: Image Classification with Hierarchical Prompting

04/13/2023
by   Wenhao Wang, et al.
0

This paper explores a hierarchical prompting mechanism for the hierarchical image classification (HIC) task. Different from prior HIC methods, our hierarchical prompting is the first to explicitly inject ancestor-class information as a tokenized hint that benefits the descendant-class discrimination. We think it well imitates human visual recognition, i.e., humans may use the ancestor class as a prompt to draw focus on the subtle differences among descendant classes. We model this prompting mechanism into a Transformer with Hierarchical Prompting (TransHP). TransHP consists of three steps: 1) learning a set of prompt tokens to represent the coarse (ancestor) classes, 2) on-the-fly predicting the coarse class of the input image at an intermediate block, and 3) injecting the prompt token of the predicted coarse class into the intermediate feature. Though the parameters of TransHP maintain the same for all input images, the injected coarse-class prompt conditions (modifies) the subsequent feature extraction and encourages a dynamic focus on relatively subtle differences among the descendant classes. Extensive experiments show that TransHP improves image classification on accuracy (e.g., improving ViT-B/16 by +2.83 efficiency (e.g., +12.69 model explainability. Moreover, TransHP also performs favorably against prior HIC methods, showing that TransHP well exploits the hierarchical information.

READ FULL TEXT
research
11/01/2021

Hierarchical Image Classification with A Literally Toy Dataset

Unsupervised domain adaptation (UDA) in image classification remains a b...
research
03/19/2021

Scalable Visual Transformers with Hierarchical Pooling

The recently proposed Visual image Transformers (ViT) with pure attentio...
research
08/19/2022

Improved Image Classification with Token Fusion

In this paper, we propose a method using the fusion of CNN and transform...
research
04/14/2006

Biologically Inspired Hierarchical Model for Feature Extraction and Localization

Feature extraction and matching are among central problems of computer v...
research
02/28/2023

Enhancing Classification with Hierarchical Scalable Query on Fusion Transformer

Real-world vision based applications require fine-grained classification...
research
12/11/2018

Coarse-to-fine: A RNN-based hierarchical attention model for vehicle re-identification

Vehicle re-identification is an important problem and becomes desirable ...
research
07/20/2020

GREEN: a Graph REsidual rE-ranking Network for Grading Diabetic Retinopathy

The automatic grading of diabetic retinopathy (DR) facilitates medical d...

Please sign up or login with your details

Forgot password? Click here to reset