Multi-Head Self-Attention via Vision Transformer for Zero-Shot Learning

07/30/2021
by   Faisal Alamri, et al.
16

Zero-Shot Learning (ZSL) aims to recognise unseen object classes, which are not observed during the training phase. The existing body of works on ZSL mostly relies on pretrained visual features and lacks the explicit attribute localisation mechanism on images. In this work, we propose an attention-based model in the problem settings of ZSL to learn attributes useful for unseen class recognition. Our method uses an attention mechanism adapted from Vision Transformer to capture and learn discriminative attributes by splitting images into small patches. We conduct experiments on three popular ZSL benchmarks (i.e., AWA2, CUB and SUN) and set new state-of-the-art harmonic mean results on all the three datasets, which illustrate the effectiveness of our proposed method.

READ FULL TEXT

page 3

page 6

research
10/02/2021

Implicit and Explicit Attention for Zero-Shot Learning

Most of the existing Zero-Shot Learning (ZSL) methods focus on learning ...
research
03/01/2019

Learning where to look: Semantic-Guided Multi-Attention Localization for Zero-Shot Learning

Zero-shot learning extends the conventional object classification to the...
research
12/16/2021

TransZero++: Cross Attribute-Guided Transformer for Zero-Shot Learning

Zero-shot learning (ZSL) tackles the novel class recognition problem by ...
research
07/12/2022

eX-ViT: A Novel eXplainable Vision Transformer for Weakly Supervised Semantic Segmentation

Recently vision transformer models have become prominent models for a ra...
research
02/02/2023

Vision Transformer-based Feature Extraction for Generalized Zero-Shot Learning

Generalized zero-shot learning (GZSL) is a technique to train a deep lea...
research
08/24/2022

Improved Zero-Shot Audio Tagging Classification with Patchout Spectrogram Transformers

Standard machine learning models for tagging and classifying acoustic si...
research
04/01/2016

How to Transfer? Zero-Shot Object Recognition via Hierarchical Transfer of Semantic Attributes

Attribute based knowledge transfer has proven very successful in visual ...

Please sign up or login with your details

Forgot password? Click here to reset