Log In Sign Up

Normalization Matters in Zero-Shot Learning

by   Ivan Skorokhodov, et al.

An ability to grasp new concepts from their descriptions is one of the key features of human intelligence, and zero-shot learning (ZSL) aims to incorporate this property into machine learning models. In this paper, we theoretically investigate two very popular tricks used in ZSL: "normalize+scale" trick and attributes normalization and show how they help to preserve a signal's variance in a typical model during a forward pass. Next, we demonstrate that these two tricks are not enough to normalize a deep ZSL network. We derive a new initialization scheme, which allows us to demonstrate strong state-of-the-art results on 4 out of 5 commonly used ZSL datasets: SUN, CUB, AwA1, and AwA2 while being on average 2 orders faster than the closest runner-up. Finally, we generalize ZSL to a broader problem – Continual Zero-Shot Learning (CZSL) and test our ideas in this new setup. The source code to reproduce all the results is available at


page 1

page 2

page 3

page 4


Dynamic VAEs with Generative Replay for Continual Zero-shot Learning

Continual zero-shot learning(CZSL) is a new domain to classify objects s...

Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learning

Zero-shot learning (ZSL) has been shown to be a promising approach to ge...

Visually Analyzing and Steering Zero Shot Learning

We propose a visual analytics system to help a user analyze and steer ze...

Geometric Generalization Based Zero-Shot Learning Dataset Infinite World: Simple Yet Powerful

Raven's Progressive Matrices are one of the widely used tests in evaluat...

ZS4IE: A toolkit for Zero-Shot Information Extraction with simple Verbalizations

The current workflow for Information Extraction (IE) analysts involves t...

Zero-Shot Multi-View Indoor Localization via Graph Location Networks

Indoor localization is a fundamental problem in location-based applicati...

A Deep Dive into Adversarial Robustness in Zero-Shot Learning

Machine learning (ML) systems have introduced significant advances in va...