Continual Learning Based on OOD Detection and Task Masking

by   Gyuhak Kim, et al.
ByteDance Inc.
University of Illinois at Chicago

Existing continual learning techniques focus on either task incremental learning (TIL) or class incremental learning (CIL) problem, but not both. CIL and TIL differ mainly in that the task-id is provided for each test sample during testing for TIL, but not provided for CIL. Continual learning methods intended for one problem have limitations on the other problem. This paper proposes a novel unified approach based on out-of-distribution (OOD) detection and task masking, called CLOM, to solve both problems. The key novelty is that each task is trained as an OOD detection model rather than a traditional supervised learning model, and a task mask is trained to protect each task to prevent forgetting. Our evaluation shows that CLOM outperforms existing state-of-the-art baselines by large margins. The average TIL/CIL accuracy of CLOM over six experiments is 87.6/67.9 only 82.4/55.0


Open-World Continual Learning: Unifying Novelty Detection and Continual Learning

As AI agents are increasingly used in the real open world with unknowns ...

A Theoretical Study on Solving Continual Learning

Continual learning (CL) learns a sequence of tasks incrementally. There ...

Taxonomic Class Incremental Learning

The problem of continual learning has attracted rising attention in rece...

CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks

This paper studies continual learning (CL) of a sequence of aspect senti...

Class-Incremental Learning based on Label Generation

Despite the great success of pre-trained language models, it is still a ...

Lifelong Learning Without a Task Oracle

Supervised deep neural networks are known to undergo a sharp decline in ...

Please sign up or login with your details

Forgot password? Click here to reset