Generative Multi-Label Zero-Shot Learning

01/27/2021
by   Akshita Gupta, et al.
10

Multi-label zero-shot learning strives to classify images into multiple unseen categories for which no data is available during training. The test samples can additionally contain seen categories in the generalized variant. Existing approaches rely on learning either shared or label-specific attention from the seen classes. Nevertheless, computing reliable attention maps for unseen classes during inference in a multi-label setting is still a challenge. In contrast, state-of-the-art single-label generative adversarial network (GAN) based approaches learn to directly synthesize the class-specific visual features from the corresponding class attribute embeddings. However, synthesizing multi-label features from GANs is still unexplored in the context of zero-shot setting. In this work, we introduce different fusion approaches at the attribute-level, feature-level and cross-level (across attribute and feature-levels) for synthesizing multi-label features from their corresponding multi-label class embedding. To the best of our knowledge, our work is the first to tackle the problem of multi-label feature synthesis in the (generalized) zero-shot setting. Comprehensive experiments are performed on three zero-shot image classification benchmarks: NUS-WIDE, Open Images and MS COCO. Our cross-level fusion-based generative approach outperforms the state-of-the-art on all three datasets. Furthermore, we show the generalization capabilities of our fusion approach in the zero-shot detection task on MS COCO, achieving favorable performance against existing methods. The source code is available at https://github.com/akshitac8/Generative_MLZSL.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/07/2018

Multi-Label Zero-Shot Learning with Transfer-Aware Label Embedding Projection

Zero-shot learning transfers knowledge from seen classes to novel unseen...
08/20/2021

Discriminative Region-based Multi-Label Zero-Shot Learning

Multi-label zero-shot learning (ZSL) is a more realistic counter-part of...
08/19/2022

A Dual Modality Approach For (Zero-Shot) Multi-Label Classification

In computer vision, multi-label classification, including zero-shot mult...
02/26/2019

LaSO: Label-Set Operations networks for multi-label few-shot learning

Example synthesis is one of the leading methods to tackle the problem of...
07/14/2021

Multi-Label Generalized Zero Shot Learning for the Classification of Disease in Chest Radiographs

Despite the success of deep neural networks in chest X-ray (CXR) diagnos...
05/28/2021

pRSL: Interpretable Multi-label Stacking by Learning Probabilistic Rules

A key task in multi-label classification is modeling the structure betwe...
08/16/2021

Structure-Aware Feature Generation for Zero-Shot Learning

Zero-Shot Learning (ZSL) targets at recognizing unseen categories by lev...

Code Repositories

Generative_MLZSL

Generative Multi-Label Zero-Shot Learning


view repo