Mitigating Generation Shifts for Generalized Zero-Shot Learning

07/07/2021
by   Zhi Chen, et al.
0

Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. It is natural to derive generative models and hallucinate training samples for unseen classes based on the knowledge learned from the seen samples. However, most of these models suffer from the `generation shifts', where the synthesized samples may drift from the real distribution of unseen data. In this paper, we conduct an in-depth analysis on this issue and propose a novel Generation Shifts Mitigating Flow (GSMFlow) framework, which is comprised of multiple conditional affine coupling layers for learning unseen data synthesis efficiently and effectively. In particular, we identify three potential problems that trigger the generation shifts, i.e., semantic inconsistency, variance decay, and structural permutation and address them respectively. First, to reinforce the correlations between the generated samples and the respective attributes, we explicitly embed the semantic information into the transformations in each of the coupling layers. Second, to recover the intrinsic variance of the synthesized unseen features, we introduce a visual perturbation strategy to diversify the intra-class variance of generated data and hereby help adjust the decision boundary of the classifier. Third, to avoid structural permutation in the semantic space, we propose a relative positioning strategy to manipulate the attribute embeddings, guiding which to fully preserve the inter-class geometric structure. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings. Our code is available at: https://github.com/uqzhichen/GSMFlow

READ FULL TEXT
research
07/05/2022

GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) aims to recognize images from both...
research
08/01/2023

Zero-Shot Learning by Harnessing Adversarial Samples

Zero-Shot Learning (ZSL) aims to recognize unseen classes by generalizin...
research
04/25/2022

Zero-Shot Logit Adjustment

Semantic-descriptor-based Generalized Zero-Shot Learning (GZSL) poses ch...
research
08/09/2020

A Boundary Based Out-of-Distribution Classifier for Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) is a challenging topic that has pr...
research
03/19/2021

DFS: A Diverse Feature Synthesis Model for Generalized Zero-Shot Learning

Generative based strategy has shown great potential in the Generalized Z...
research
03/01/2021

Counterfactual Zero-Shot and Open-Set Visual Recognition

We present a novel counterfactual framework for both Zero-Shot Learning ...
research
09/07/2023

Zero-Shot Scene Graph Generation via Triplet Calibration and Reduction

Scene Graph Generation (SGG) plays a pivotal role in downstream vision-l...

Please sign up or login with your details

Forgot password? Click here to reset