Energy-based Latent Aligner for Incremental Learning

03/28/2022
by   K J Joseph, et al.
8

Deep learning models tend to forget their earlier knowledge while incrementally learning new tasks. This behavior emerges because the parameter updates optimized for the new tasks may not align well with the updates suitable for older tasks. The resulting latent representation mismatch causes forgetting. In this work, we propose ELI: Energy-based Latent Aligner for Incremental Learning, which first learns an energy manifold for the latent representations such that previous task latents will have low energy and the current task latents have high energy values. This learned manifold is used to counter the representational shift that happens during incremental learning. The implicit regularization that is offered by our proposed methodology can be used as a plug-and-play module in existing incremental learning methodologies. We validate this through extensive evaluation on CIFAR-100, ImageNet subset, ImageNet 1k and Pascal VOC datasets. We observe consistent improvement when ELI is added to three prominent methodologies in class-incremental learning, across multiple incremental settings. Further, when added to the state-of-the-art incremental object detector, ELI provides over 5 accuracy, corroborating its effectiveness and complementary advantage to existing art.

READ FULL TEXT

page 7

page 11

page 12

research
02/03/2019

Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence

Incremental learning suffers from two challenging problems; forgetting o...
research
06/17/2022

TKIL: Tangent Kernel Approach for Class Balanced Incremental Learning

When learning new tasks in a sequential manner, deep neural networks ten...
research
03/23/2023

Adaptive Regularization for Class-Incremental Learning

Class-Incremental Learning updates a deep classifier with new categories...
research
11/23/2016

iCaRL: Incremental Classifier and Representation Learning

A major open problem on the road to artificial intelligence is the devel...
research
04/28/2020

Small-Task Incremental Learning

Lifelong learning has attracted much attention, but existing works still...
research
01/30/2018

Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

We study the incremental learning problem for the classification task, a...
research
06/22/2018

Continuous Learning in Single-Incremental-Task Scenarios

It was recently shown that architectural, regularization and rehearsal s...

Please sign up or login with your details

Forgot password? Click here to reset