Classifier Crafting: Turn Your ConvNet into a Zero-Shot Learner!

03/20/2021
by   Jacopo Cavazza, et al.
74

In Zero-shot learning (ZSL), we classify unseen categories using textual descriptions about their expected appearance when observed (class embeddings) and a disjoint pool of seen classes, for which annotated visual data are accessible. We tackle ZSL by casting a "vanilla" convolutional neural network (e.g. AlexNet, ResNet-101, DenseNet-201 or DarkNet-53) into a zero-shot learner. We do so by crafting the softmax classifier: we freeze its weights using fixed seen classification rules, either semantic (seen class embeddings) or visual (seen class prototypes). Then, we learn a data-driven and ZSL-tailored feature representation on seen classes only to match these fixed classification rules. Given that the latter seamlessly generalize towards unseen classes, while requiring not actual unseen data to be computed, we can perform ZSL inference by augmenting the pool of classification rules at test time while keeping the very same representation we learnt: nowhere re-training or fine-tuning on unseen data is performed. The combination of semantic and visual crafting (by simply averaging softmax scores) improves prior state-of-the-art methods in benchmark datasets for standard, inductive ZSL. After rebalancing predictions to better handle the joint inference over seen and unseen classes, we outperform prior generalized, inductive ZSL methods as well. Also, we gain interpretability at no additional cost, by using neural attention methods (e.g., grad-CAM) as they are. Code will be made publicly available.

READ FULL TEXT

page 1

page 5

page 6

page 8

research
09/08/2019

Meta-Transfer Networks for Zero-Shot Learning

Zero-Shot Learning (ZSL) aims at recognizing unseen categories using som...
research
08/14/2019

Generalised Zero-Shot Learning with Domain Classification in a Joint Semantic and Visual Space

Generalised zero-shot learning (GZSL) is a classification problem where ...
research
09/10/2019

Semantic Similarity Based Softmax Classifier for Zero-Shot Learning

Zero-Shot Learning (ZSL) is a classification task where we do not have e...
research
05/09/2020

Generative Model-driven Structure Aligning Discriminative Embeddings for Transductive Zero-shot Learning

Zero-shot Learning (ZSL) is a transfer learning technique which aims at ...
research
03/27/2019

Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration

Zero-shot learning (ZSL) for image classification focuses on recognizing...
research
06/14/2018

Nearly Zero-Shot Learning for Semantic Decoding in Spoken Dialogue Systems

This paper presents two ways of dealing with scarce data in semantic dec...
research
07/18/2017

One-Shot Learning in Discriminative Neural Networks

We consider the task of one-shot learning of visual categories. In this ...

Please sign up or login with your details

Forgot password? Click here to reset