A Novel Perspective to Zero-shot Learning: Towards an Alignment of Manifold Structures via Semantic Feature Expansion

04/30/2020
by   Jingcai Guo, et al.
6

Zero-shot learning aims at recognizing unseen classes (no training example) with knowledge transferred from seen classes. This is typically achieved by exploiting a semantic feature space shared by both seen and unseen classes, i.e., attribute or word vector, as the bridge. One common practice in zero-shot learning is to train a projection between the visual and semantic feature spaces with labeled seen classes examples. When inferring, this learned projection is applied to unseen classes and recognizes the class labels by some metrics. However, the visual and semantic feature spaces are mutually independent and have quite different manifold structures. Under such a paradigm, most existing methods easily suffer from the domain shift problem and weaken the performance of zero-shot recognition. To address this issue, we propose a novel model called AMS-SFE. It considers the alignment of manifold structures by semantic feature expansion. Specifically, we build upon an autoencoder-based model to expand the semantic features from the visual inputs. Additionally, the expansion is jointly guided by an embedded manifold extracted from the visual feature space of the data. Our model is the first attempt to align both feature spaces by expanding semantic features and derives two benefits: first, we expand some auxiliary features that enhance the semantic feature space; second and more importantly, we implicitly align the manifold structures between the visual and semantic feature spaces; thus, the projection can be better trained and mitigate the domain shift problem. Extensive experiments show significant performance improvement, which verifies the effectiveness of our model.

READ FULL TEXT

page 1

page 5

page 8

page 10

page 11

research
04/12/2019

AMS-SFE: Towards an Alignment of Manifold Structures via Semantic Feature Expansion for Zero-shot Learning

Zero-shot learning (ZSL) aims at recognizing unseen classes with knowled...
research
04/12/2021

Learning Robust Visual-semantic Mapping for Zero-shot Learning

Zero-shot learning (ZSL) aims at recognizing unseen class examples (e.g....
research
03/23/2021

Expanding Semantic Knowledge for Zero-shot Graph Embedding

Zero-shot graph embedding is a major challenge for supervised graph lear...
research
03/30/2019

Adaptive Adjustment with Semantic Feature Space for Zero-Shot Recognition

In most recent years, zero-shot recognition (ZSR) has gained increasing ...
research
07/27/2023

TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentation

Recent label mix-based augmentation methods have shown their effectivene...
research
06/26/2023

An Integral Projection-based Semantic Autoencoder for Zero-Shot Learning

Zero-shot Learning (ZSL) classification categorizes or predicts classes ...
research
01/19/2015

Transductive Multi-view Zero-Shot Learning

Most existing zero-shot learning approaches exploit transfer learning vi...

Please sign up or login with your details

Forgot password? Click here to reset