DER: Dynamically Expandable Representation for Class Incremental Learning

03/31/2021
by   Shipeng Yan, et al.
0

We address the problem of class incremental learning, which is a core step towards achieving adaptive vision intelligence. In particular, we consider the task setting of incremental learning with limited memory and aim to achieve better stability-plasticity trade-off. To this end, we propose a novel two-stage learning approach that utilizes a dynamically expandable representation for more effective incremental concept modeling. Specifically, at each incremental step, we freeze the previously learned representation and augment it with additional feature dimensions from a new learnable feature extractor. This enables us to integrate new visual concepts with retaining learned knowledge. We dynamically expand the representation according to the complexity of novel concepts by introducing a channel-level mask-based pruning strategy. Moreover, we introduce an auxiliary loss to encourage the model to learn diverse and discriminate features for novel concepts. We conduct extensive experiments on the three class incremental learning benchmarks and our method consistently outperforms other methods with a large margin.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/08/2022

General Incremental Learning with Domain-aware Categorical Representations

Continual learning is an important problem for achieving human-level int...
research
08/06/2021

Incremental Feature Learning For Infinite Data

This study addresses the actual behavior of the credit-card fraud detect...
research
03/23/2023

Adaptive Regularization for Class-Incremental Learning

Class-Incremental Learning updates a deep classifier with new categories...
research
07/14/2022

E2-AEN: End-to-End Incremental Learning with Adaptively Expandable Network

Expandable networks have demonstrated their advantages in dealing with c...
research
04/04/2023

On the Stability-Plasticity Dilemma of Class-Incremental Learning

A primary goal of class-incremental learning is to strike a balance betw...
research
02/24/2020

Mnemonics Training: Multi-Class Incremental Learning without Forgetting

Multi-Class Incremental Learning (MCIL) aims to learn new concepts by in...
research
10/13/2022

ALIFE: Adaptive Logit Regularizer and Feature Replay for Incremental Semantic Segmentation

We address the problem of incremental semantic segmentation (ISS) recogn...

Please sign up or login with your details

Forgot password? Click here to reset