A general approach to progressive intelligence

04/27/2020
by   Joshua T. Vogelstein, et al.
42

In biological learning, data is used to improve performance on the task at hand, while simultaneously improving performance on both previously encountered tasks and as yet unconsidered future tasks. In contrast, classical machine learning starts from a blank slate, or tabula rasa, using data only for the single task at hand. While typical transfer learning algorithms can improve performance on future tasks, their performance degrades upon learning new tasks. Many recent approaches have attempted to mitigate this issue, called catastrophic forgetting, to maintain performance given new tasks. But striving to avoid forgetting sets the goal unnecessarily low: the goal of progressive learning, whether biological or artificial, is to improve performance on all tasks (including past and future) with any new data. We propose a general approach to progressive learning that ensembles representations, rather than learners. We show that ensembling representations—including representations learned by decision forests or neural networks—enables both forward and backward transfer on a variety of simulated and real data tasks, including vision, language, and adversarial tasks. This work suggests that further improvements in progressive learning may follow from a deeper understanding of how biological learning achieves such high degrees of efficiency.

READ FULL TEXT

page 4

page 5

page 6

page 7

page 8

page 11

page 14

research
04/27/2020

A general approach to progressive learning

In biological learning, data is used to improve performance on the task ...
research
12/02/2016

Overcoming catastrophic forgetting in neural networks

The ability to learn tasks in a sequential fashion is crucial to the dev...
research
01/17/2023

Artificial Neuronal Ensembles with Learned Context Dependent Gating

Biological neural networks are capable of recruiting different sets of n...
research
12/24/2022

Utilizing Priming to Identify Optimal Class Ordering to Alleviate Catastrophic Forgetting

In order for artificial neural networks to begin accurately mimicking bi...
research
02/22/2019

Generative Memory for Lifelong Reinforcement Learning

Our research is focused on understanding and applying biological memory ...
research
09/23/2021

The Role of Bio-Inspired Modularity in General Learning

One goal of general intelligence is to learn novel information without o...
research
11/28/2022

Progressive Learning without Forgetting

Learning from changing tasks and sequential experience without forgettin...

Please sign up or login with your details

Forgot password? Click here to reset