On the Transferability of Visual Features in Generalized Zero-Shot Learning

11/22/2022
by   Paola Cascante-Bonilla, et al.
0

Generalized Zero-Shot Learning (GZSL) aims to train a classifier that can generalize to unseen classes, using a set of attributes as auxiliary information, and the visual features extracted from a pre-trained convolutional neural network. While recent GZSL methods have explored various techniques to leverage the capacity of these features, there has been an extensive growth of representation learning techniques that remain under-explored. In this work, we investigate the utility of different GZSL methods when using different feature extractors, and examine how these models' pre-training objectives, datasets, and architecture design affect their feature representation ability. Our results indicate that 1) methods using generative components for GZSL provide more advantages when using recent feature extractors; 2) feature extractors pre-trained using self-supervised learning objectives and knowledge distillation provide better feature representations, increasing up to 15 performance when used with recent GZSL techniques; 3) specific feature extractors pre-trained with larger datasets do not necessarily boost the performance of GZSL methods. In addition, we investigate how GZSL methods fare against CLIP, a more recent multi-modal pre-trained model with strong zero-shot performance. We found that GZSL tasks still benefit from generative-based GZSL methods along with CLIP's internet-scale pre-training to achieve state-of-the-art performance in fine-grained datasets. We release a modular framework for analyzing representation learning issues in GZSL here: https://github.com/uvavision/TV-GZSL

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/20/2022

Temporal and cross-modal attention for audio-visual zero-shot learning

Audio-visual generalised zero-shot learning for video classification req...
research
11/28/2022

SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification

Although significant progress has been made in few-shot learning, most o...
research
02/23/2021

Multi-Knowledge Fusion for New Feature Generation in Generalized Zero-Shot Learning

Suffering from the semantic insufficiency and domain-shift problems, mos...
research
03/27/2023

Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning

Recent compositional zero-shot learning (CZSL) methods adapt pre-trained...
research
07/15/2020

Enhancing Generalized Zero-Shot Learning via Adversarial Visual-Semantic Interaction

The performance of generative zero-shot methods mainly depends on the qu...
research
10/09/2021

ZSpeedL – Evaluating the Performance of Zero-Shot Learning Methods using Low-Power Devices

The recognition of unseen objects from a semantic representation or text...
research
12/20/2019

Locality and compositionality in zero-shot learning

In this work we study locality and compositionality in the context of le...

Please sign up or login with your details

Forgot password? Click here to reset