Incremental Few-Shot Learning with Attention Attractor Networks

by   Mengye Ren, et al.

Machine learning classifiers are often trained to recognize a set of pre-defined classes. However, in many real applications, it is often desirable to have the flexibility of learning additional concepts, without re-training on the full training set. This paper addresses this problem, incremental few-shot learning, where a regular classification network has already been trained to recognize a set of base classes; and several extra novel classes are being considered, each with only a few labeled examples. After learning the novel classes, the model is then evaluated on the overall performance of both base and novel classes. To this end, we propose a meta-learning model, the Attention Attractor Network, which regularizes the learning of novel classes. In each episode, we train a set of new weights to recognize novel classes until they converge, and we show that the technique of recurrent back-propagation can back-propagate through the optimization process and facilitate the learning of the attractor network regularizer. We demonstrate that the learned attractor network can recognize novel classes while remembering old classes without the need to review the original training set, outperforming baselines that do not rely on an iterative optimization process.


page 1

page 2

page 3

page 4


Coarse-To-Fine Incremental Few-Shot Learning

Different from fine-tuning models pre-trained on a large-scale dataset o...

Generative Low-Shot Network Expansion

Conventional deep learning classifiers are static in the sense that they...

XtarNet: Learning to Extract Task-Adaptive Representation for Incremental Few-Shot Learning

Learning novel concepts while preserving prior knowledge is a long-stand...

Label Hallucination for Few-Shot Classification

Few-shot classification requires adapting knowledge learned from a large...

Transductive Propagation Network for Few-shot Learning

Few-shot learning aims to build a learner that quickly generalizes to no...

Few-shot Class-incremental Audio Classification Using Adaptively-refined Prototypes

New classes of sounds constantly emerge with a few samples, making it ch...

Associative Alignment for Few-shot Image Classification

Few-shot image classification aims at training a model by using only a f...

Please sign up or login with your details

Forgot password? Click here to reset