Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis

09/04/2019
by   Tyler L. Hayes, et al.
19

When a robot acquires new information, ideally it would immediately be capable of using that information to understand its environment. While deep neural networks are now widely used by robots for inferring semantic information, conventional neural networks suffer from catastrophic forgetting when they are incrementally updated, with new knowledge overwriting established representations. While a variety of approaches have been developed that attempt to mitigate catastrophic forgetting in the incremental batch learning scenario, in which an agent learns a large collection of labeled samples at once, streaming learning has been much less studied in the robotics and deep learning communities. In streaming learning, an agent learns instances one-by-one and can be tested at any time. Here, we revisit streaming linear discriminant analysis, which has been widely used in the data mining research community. By combining streaming linear discriminant analysis with deep learning, we are able to outperform both incremental batch learning and streaming learning algorithms on both ImageNet-1K and CORe50.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/15/2023

Continual Learning with Deep Streaming Regularized Discriminant Analysis

Continual learning is increasingly sought after in real world machine le...
research
09/16/2018

Memory Efficient Experience Replay for Streaming Learning

In supervised machine learning, an agent is typically trained once and t...
research
11/09/2022

Cold Start Streaming Learning for Deep Networks

The ability to dynamically adapt neural networks to newly-available data...
research
02/11/2020

Incremental Fast Subclass Discriminant Analysis

This paper proposes an incremental solution to Fast Subclass Discriminan...
research
10/06/2019

REMIND Your Neural Network to Prevent Catastrophic Forgetting

In lifelong machine learning, a robotic agent must be incrementally upda...
research
05/28/2023

Backdoor Attacks Against Incremental Learners: An Empirical Evaluation Study

Large amounts of incremental learning algorithms have been proposed to a...
research
10/01/2020

StreamSoNG: A Soft Streaming Classification Approach

Examining most streaming clustering algorithms leads to the understandin...

Please sign up or login with your details

Forgot password? Click here to reset