Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders

12/05/2018
by   Edgar Schönfeld, et al.
0

Many approaches in generalized zero-shot learning rely on cross-modal mapping between the image feature space and the class embedding space. As labeled images are rare, one direction is to augment the dataset by generating either images or image features. However, the former misses fine-grained details and the latter requires learning a mapping associated with class embeddings. In this work, we take feature generation one step further and propose a model where a shared latent space of image features and class embeddings is learned by modality-specific aligned variational autoencoders. This leaves us with the required discriminative information about the image and classes in the latent features, on which we train a softmax classifier. The key to our approach is that we align the distributions learned from images and from side-information to construct latent features that contain the essential multi-modal information associated with unseen classes. We evaluate our learned latent features on several benchmark datasets, i.e. CUB, SUN, AWA1 and AWA2, and establish a new state-of-the-art on generalized zero-shot as well as on few-shot learning. Moreover, our results on ImageNet with various zero-shot splits show that our latent features generalize well in large-scale settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/27/2020

AVGZSLNet: Audio-Visual Generalized Zero-Shot Learning by Reconstructing Label Features from Multi-Modal Embeddings

In this paper, we solve for the problem of generalized zero-shot learnin...
research
06/26/2021

Generalized Zero-Shot Learning using Multimodal Variational Auto-Encoder with Semantic Concepts

With the ever-increasing amount of data, the central challenge in multim...
research
12/24/2021

Learning Aligned Cross-Modal Representation for Generalized Zero-Shot Classification

Learning a common latent embedding by aligning the latent spaces of cros...
research
03/25/2019

f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning

When labeled training data is scarce, a promising data augmentation appr...
research
03/29/2016

Latent Embeddings for Zero-shot Classification

We present a novel latent embedding model for learning a compatibility f...
research
07/15/2020

Enhancing Generalized Zero-Shot Learning via Adversarial Visual-Semantic Interaction

The performance of generative zero-shot methods mainly depends on the qu...
research
07/20/2020

Fantastic Embeddings and How to Align Them: Zero-Shot Inference in a Multi-Shop Scenario

This paper addresses the challenge of leveraging multiple embedding spac...

Please sign up or login with your details

Forgot password? Click here to reset