Target Layer Regularization for Continual Learning Using Cramer-Wold Generator

11/15/2021
by   Marcin Mazur, et al.
0

We propose an effective regularization strategy (CW-TaLaR) for solving continual learning problems. It uses a penalizing term expressed by the Cramer-Wold distance between two probability distributions defined on a target layer of an underlying neural network that is shared by all tasks, and the simple architecture of the Cramer-Wold generator for modeling output data representation. Our strategy preserves target layer distribution while learning a new task but does not require remembering previous tasks' datasets. We perform experiments involving several common supervised frameworks, which prove the competitiveness of the CW-TaLaR method in comparison to a few existing state-of-the-art continual learning models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/24/2020

Energy-Based Models for Continual Learning

We motivate Energy-Based Models (EBMs) as a promising model class for co...
research
07/27/2021

Continual Learning with Neuron Activation Importance

Continual learning is a concept of online learning with multiple sequent...
research
12/06/2019

Regularization Shortcomings for Continual Learning

In classical machine learning, the data streamed to the algorithms is as...
research
06/26/2021

Continual Learning via Inter-Task Synaptic Mapping

Learning from streaming tasks leads a model to catastrophically erase un...
research
05/26/2022

The Effect of Task Ordering in Continual Learning

We investigate the effect of task ordering on continual learning perform...
research
06/12/2020

CPR: Classifier-Projection Regularization for Continual Learning

We propose a general, yet simple patch that can be applied to existing r...
research
03/18/2022

Class-wise Classifier Design Capable of Continual Learning using Adaptive Resonance Theory-based Topological Clustering

This paper proposes a supervised classification algorithm capable of con...

Please sign up or login with your details

Forgot password? Click here to reset