Better Knowledge Retention through Metric Learning

11/26/2020
by   Ke Li, et al.
4

In continual learning, new categories may be introduced over time, and an ideal learning system should perform well on both the original categories and the new categories. While deep neural nets have achieved resounding success in the classical supervised setting, they are known to forget about knowledge acquired in prior episodes of learning if the examples encountered in the current episode of learning are drastically different from those encountered in prior episodes. In this paper, we propose a new method that can both leverage the expressive power of deep neural nets and is resilient to forgetting when new categories are introduced. We found the proposed method can reduce forgetting by 2.3x to 6.9x on CIFAR-10 compared to existing methods and by 1.8x to 2.7x on ImageNet compared to an oracle baseline.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/02/2023

Online Continual Learning via the Knowledge Invariant and Spread-out Properties

The goal of continual learning is to provide intelligent agents that are...
research
09/29/2022

RECALL: Rehearsal-free Continual Learning for Object Classification

Convolutional neural networks show remarkable results in classification ...
research
04/22/2020

Continual Learning of Object Instances

We propose continual instance learning - a method that applies the conce...
research
08/14/2021

Weakly Supervised Continual Learning

Continual Learning (CL) investigates how to train Deep Networks on a str...
research
06/12/2020

Move-to-Data: A new Continual Learning approach with Deep CNNs, Application for image-class recognition

In many real-life tasks of application of supervised learning approaches...
research
11/30/2021

Hierarchical Prototype Networks for Continual Graph Representation Learning

Despite significant advances in graph representation learning, little at...
research
09/05/2023

Continual Improvement of Threshold-Based Novelty Detection

When evaluated in dynamic, open-world situations, neural networks strugg...

Please sign up or login with your details

Forgot password? Click here to reset