Continual Learning Through Synaptic Intelligence

03/13/2017
by   Friedemann Zenke, et al.
0

While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks continually adapt to changing domains, possibly by leveraging complex molecular machinery to solve many tasks simultaneously. In this study, we introduce intelligent synapses that bring some of this biological complexity into artificial neural networks. Each synapse accumulates task relevant information over time, and exploits this information to rapidly store new memories without forgetting old ones. We evaluate our approach on continual learning of classification tasks, and show that it dramatically reduces forgetting while maintaining computational efficiency.

READ FULL TEXT
research
10/23/2021

AFEC: Active Forgetting of Negative Transfer in Continual Learning

Continual learning aims to learn a sequence of tasks from dynamic data d...
research
04/30/2020

Importance Driven Continual Learning for Segmentation Across Domains

The ability of neural networks to continuously learn and adapt to new ta...
research
09/09/2022

Continual learning benefits from multiple sleep mechanisms: NREM, REM, and Synaptic Downscaling

Learning new tasks and skills in succession without losing prior learnin...
research
06/01/2023

Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift

Continual learning (CL) is an important technique to allow artificial ne...
research
11/23/2022

Cooperative data-driven modeling

Data-driven modeling in mechanics is evolving rapidly based on recent ma...
research
04/05/2019

Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning

Models trained in the context of continual learning (CL) should be able ...
research
09/29/2022

RECALL: Rehearsal-free Continual Learning for Object Classification

Convolutional neural networks show remarkable results in classification ...

Please sign up or login with your details

Forgot password? Click here to reset