Self-recovery of memory via generative replay

01/15/2023
by   Zhenglong Zhou, et al.
0

A remarkable capacity of the brain is its ability to autonomously reorganize memories during offline periods. Memory replay, a mechanism hypothesized to underlie biological offline learning, has inspired offline methods for reducing forgetting in artificial neural networks in continual learning settings. A memory-efficient and neurally-plausible method is generative replay, which achieves state of the art performance on continual learning benchmarks. However, unlike the brain, standard generative replay does not self-reorganize memories when trained offline on its own replay samples. We propose a novel architecture that augments generative replay with an adaptive, brain-like capacity to autonomously recover memories. We demonstrate this capacity of the architecture across several continual learning tasks and environments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/11/2019

Online Continual Learning with Maximally Interfered Retrieval

Continual learning, the setting where a learning agent is faced with a n...
research
08/04/2022

A Benchmark and Empirical Analysis for Replay Strategies in Continual Learning

With the capacity of continual learning, humans can continuously acquire...
research
10/29/2018

Marginal Replay vs Conditional Replay for Continual Learning

We present a new replay-based method of continual classification learnin...
research
09/21/2021

Learning offline: memory replay in biological and artificial reinforcement learning

Learning to act in an environment to maximise rewards is among the brain...
research
05/29/2023

SHARP: Sparsity and Hidden Activation RePlay for Neuro-Inspired Continual Learning

Deep neural networks (DNNs) struggle to learn in dynamic environments si...
research
10/26/2021

Brain-inspired feature exaggeration in generative replay for continual learning

The catastrophic forgetting of previously learnt classes is one of the m...
research
07/29/2023

Continual Learning in Predictive Autoscaling

Predictive Autoscaling is used to forecast the workloads of servers and ...

Please sign up or login with your details

Forgot password? Click here to reset