DeepAI
Log In Sign Up

Learnable Expansion-and-Compression Network for Few-shot Class-Incremental Learning

04/06/2021
by   Boyu Yang, et al.
0

Few-shot class-incremental learning (FSCIL), which targets at continuously expanding model's representation capacity under few supervisions, is an important yet challenging problem. On the one hand, when fitting new tasks (novel classes), features trained on old tasks (old classes) could significantly drift, causing catastrophic forgetting. On the other hand, training the large amount of model parameters with few-shot novel-class examples leads to model over-fitting. In this paper, we propose a learnable expansion-and-compression network (LEC-Net), with the aim to simultaneously solve catastrophic forgetting and model over-fitting problems in a unified framework. By tentatively expanding network nodes, LEC-Net enlarges the representation capacity of features, alleviating feature drift of old network from the perspective of model regularization. By compressing the expanded network nodes, LEC-Net purses minimal increase of model parameters, alleviating over-fitting of the expanded network from a perspective of compact representation. Experiments on the CUB/CIFAR-100 datasets show that LEC-Net improves the baseline by 5 7 LEC-Net also demonstrates the potential to be a general incremental learning approach with dynamic model expansion capability.

READ FULL TEXT

page 1

page 6

10/01/2022

Learnable Distribution Calibration for Few-Shot Class-Incremental Learning

Few-shot class-incremental learning (FSCIL) faces challenges of memorizi...
08/11/2022

Memorizing Complementation Network for Few-Shot Class-Incremental Learning

Few-shot Class-Incremental Learning (FSCIL) aims at learning new concept...
06/28/2020

Few-Shot Class-Incremental Learning via Feature Space Composition

As a challenging problem in machine learning, few-shot class-incremental...
07/26/2022

Incremental Few-Shot Semantic Segmentation via Embedding Adaptive-Update and Hyper-class Representation

Incremental few-shot semantic segmentation (IFSS) targets at incremental...
08/31/2020

Learning Adaptive Embedding Considering Incremental Class

Class-Incremental Learning (CIL) aims to train a reliable model with the...
03/19/2020

Lifelong Learning with Searchable Extension Units

Lifelong learning remains an open problem. One of its main difficulties ...
05/27/2022

Geometer: Graph Few-Shot Class-Incremental Learning via Prototype Representation

With the tremendous expansion of graphs data, node classification shows ...