Sequential mastery of multiple tasks: Networks naturally learn to learn

05/26/2019
by   Guy Davidson, et al.
0

We explore the behavior of a standard convolutional neural net in a setting that introduces classification tasks sequentially and requires the net to master new tasks while preserving mastery of previously learned tasks. This setting corresponds to that which human learners face as they acquire domain expertise, for example, as an individual reads a textbook chapter-by-chapter. Through simulations involving sequences of ten related tasks, we find reason for optimism that nets will scale well as they advance from having a single skill to becoming domain experts. We observed two key phenomena. First, _forward facilitation_---the accelerated learning of task n+1 having learned n previous tasks---grows with n. Second, _backward interference_---the forgetting of the n previous tasks when learning task n+1---diminishes with n. Amplifying forward facilitation is the goal of research on metalearning, and attenuating backward interference is the goal of research on catastrophic forgetting. We find that both of these goals are attained simply through broader exposure to a domain.

READ FULL TEXT
research
10/24/2018

Continual Classification Learning Using Generative Models

Continual learning is the ability to sequentially learn over time by acc...
research
11/28/2017

Block Neural Network Avoids Catastrophic Forgetting When Learning Multiple Task

In the present work we propose a Deep Feed Forward network architecture ...
research
12/02/2016

Overcoming catastrophic forgetting in neural networks

The ability to learn tasks in a sequential fashion is crucial to the dev...
research
04/06/2017

Encoder Based Lifelong Learning

This paper introduces a new lifelong learning solution where a single mo...
research
04/28/2020

Pseudo Rehearsal using non photo-realistic images

Deep Neural networks forget previously learnt tasks when they are faced ...
research
09/07/2018

HC-Net: Memory-based Incremental Dual-Network System for Continual learning

Training a neural network for a classification task typically assumes th...

Please sign up or login with your details

Forgot password? Click here to reset