DeepAI AI Chat
Log In Sign Up

Random Path Selection for Incremental Learning

06/03/2019
by   Jathushan Rajasegaran, et al.
0

Incremental life-long learning is a main challenge towards the long-standing goal of Artificial General Intelligence. In real-life settings, learning tasks arrive in a sequence and machine learning models must continually learn to increment already acquired knowledge. Existing incremental learning approaches, fall well below the state-of-the-art cumulative models that use all training classes at once. In this paper, we propose a random path selection algorithm, called RPSnet, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing and reuse. Our approach avoids the overhead introduced by computationally expensive evolutionary and reinforcement learning based path selection strategies while achieving considerable performance gains. As an added novelty, the proposed model integrates knowledge distillation and retrospection along with the path selection strategy to overcome catastrophic forgetting. In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity. Through extensive experiments, we demonstrate that the proposed method surpasses the state-of-the-art performance on incremental learning and by utilizing parallel computation this method can run in constant time with nearly the same efficiency as a conventional deep convolutional neural network.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/09/2023

SRIL: Selective Regularization for Class-Incremental Learning

Human intelligence gradually accepts new information and accumulates kno...
04/17/2021

On Learning the Geodesic Path for Incremental Learning

Neural networks notoriously suffer from the problem of catastrophic forg...
07/17/2022

Learning with Recoverable Forgetting

Life-long learning aims at learning a sequence of tasks without forgetti...
06/01/2023

Teacher Agent: A Non-Knowledge Distillation Method for Rehearsal-based Video Incremental Learning

With the rise in popularity of video-based social media, new categories ...
12/24/2022

Utilizing Priming to Identify Optimal Class Ordering to Alleviate Catastrophic Forgetting

In order for artificial neural networks to begin accurately mimicking bi...
01/18/2021

Incremental Knowledge Based Question Answering

In the past years, Knowledge-Based Question Answering (KBQA), which aims...
04/10/2022

FOSTER: Feature Boosting and Compression for Class-Incremental Learning

The ability to learn new concepts continually is necessary in this ever-...