Task-agnostic Continual Learning with Hybrid Probabilistic Models

06/24/2021
by   Polina Kirichenko, et al.
0

Learning new tasks continuously without forgetting on a constantly changing data distribution is essential for real-world problems but extremely challenging for modern deep learning. In this work we propose HCL, a Hybrid generative-discriminative approach to Continual Learning for classification. We model the distribution of each task and each class with a normalizing flow. The flow is used to learn the data distribution, perform classification, identify task changes, and avoid forgetting, all leveraging the invertibility and exact likelihood which are uniquely enabled by the normalizing flow model. We use the generative capabilities of the flow to avoid catastrophic forgetting through generative replay and a novel functional regularization technique. For task identification, we use state-of-the-art anomaly detection techniques based on measuring the typicality of the model's statistics. We demonstrate the strong performance of HCL on a range of continual learning benchmarks such as split-MNIST, split-CIFAR, and SVHN-MNIST.

READ FULL TEXT

Authors

page 7

05/17/2021

Shared and Private VAEs with Generative Replay for Continual Learning

Continual learning tries to learn new tasks without forgetting previousl...
06/30/2020

Enabling Continual Learning with Differentiable Hebbian Plasticity

Continual learning is the problem of sequentially learning new tasks or ...
06/24/2021

Continual Competitive Memory: A Neural System for Online Task-Free Lifelong Learning

In this article, we propose a novel form of unsupervised learning, conti...
10/18/2021

Dendritic Self-Organizing Maps for Continual Learning

Current deep learning architectures show remarkable performance when tra...
04/19/2021

Continual Learning with Fully Probabilistic Models

We present an approach for continual learning (CL) that is based on full...
02/16/2021

Adversarial Targeted Forgetting in Regularization and Generative Based Continual Learning Models

Continual (or "incremental") learning approaches are employed when addit...
03/13/2017

Continual Learning Through Synaptic Intelligence

While deep learning has led to remarkable advances across diverse applic...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.