Visual and Semantic Prototypes-Jointly Guided CNN for Generalized Zero-shot Learning

08/12/2019
by   Chuanxing Geng, et al.
10

In the process of exploring the world, the curiosity constantly drives humans to cognize new things. Supposing you are a zoologist, for a presented animal image, you can recognize it immediately if you know its class. Otherwise, you would more likely attempt to cognize it by exploiting the side-information (e.g., semantic information, etc.) you have accumulated. Inspired by this, this paper decomposes the generalized zero-shot learning (G-ZSL) task into an open set recognition (OSR) task and a zero-shot learning (ZSL) task, where OSR recognizes seen classes (if we have seen (or known) them) and rejects unseen classes (if we have never seen (or known) them before), while ZSL identifies the unseen classes rejected by the former. Simultaneously, without violating OSR's assumptions (only known class knowledge is available in training), we also first attempt to explore a new generalized open set recognition (G-OSR) by introducing the accumulated side-information from known classes to OSR. For G-ZSL, such a decomposition essentially solves the class overfitting problem with easily misclassifying unseen classes as seen classes. The problem is ubiquitous in most existing G-ZSL methods. On the other hand, for G-OSR, introducing such semantic information of known classes not only improves the recognition performance but also endows OSR with the cognitive ability of unknown classes. Specifically, a visual and semantic prototypes-jointly guided convolutional neural network (VSG-CNN) is proposed to fulfill these two tasks (G-ZSL and G-OSR) in a unified end-to-end learning framework. Extensive experiments on benchmark datasets demonstrate the advantages of our learning framework.

READ FULL TEXT

page 1

page 8

research
06/22/2018

Global Semantic Consistency for Zero-Shot Learning

In image recognition, there are many cases where training samples cannot...
research
09/26/2018

From Classical to Generalized Zero-Shot Learning: a Simple Adaptation Process

Zero-shot learning (ZSL) is concerned with the recognition of previously...
research
09/29/2020

A Prototype-Based Generalized Zero-Shot Learning Framework for Hand Gesture Recognition

Hand gesture recognition plays a significant role in human-computer inte...
research
12/03/2015

Prototypical Priors: From Improving Classification to Zero-Shot Learning

Recent works on zero-shot learning make use of side information such as ...
research
03/01/2021

Counterfactual Zero-Shot and Open-Set Visual Recognition

We present a novel counterfactual framework for both Zero-Shot Learning ...
research
11/03/2021

An Entropy-guided Reinforced Partial Convolutional Network for Zero-Shot Learning

Zero-Shot Learning (ZSL) aims to transfer learned knowledge from observe...
research
10/17/2018

Learning to Separate Domains in Generalized Zero-Shot and Open Set Learning: a probabilistic perspective

This paper studies the problem of domain division problem which aims to ...

Please sign up or login with your details

Forgot password? Click here to reset