Recent Advances of Continual Learning in Computer Vision: An Overview

09/23/2021
by   Haoxuan Qu, et al.
52

In contrast to batch learning where all training data is available at once, continual learning represents a family of methods that accumulate knowledge and learn continuously with data available in sequential order. Similar to the human learning process with the ability of learning, fusing, and accumulating new knowledge coming at different time steps, continual learning is considered to have high practical significance. Hence, continual learning has been studied in various artificial intelligence tasks. In this paper, we present a comprehensive review of the recent progress of continual learning in computer vision. In particular, the works are grouped by their representative techniques, including regularization, knowledge distillation, memory, generative replay, parameter isolation, and a combination of the above techniques. For each category of these techniques, both its characteristics and applications in computer vision are presented. At the end of this overview, several subareas, where continuous knowledge accumulation is potentially helpful while continual learning has not been well studied, are discussed.

READ FULL TEXT
research
07/11/2020

Batch-level Experience Replay with Review for Continual Learning

Continual learning is a branch of deep learning that seeks to strike a b...
research
08/28/2023

Continual Learning with Dynamic Sparse Training: Exploring Algorithms for Effective Model Updates

Continual learning (CL) refers to the ability of an intelligent system t...
research
03/31/2022

A Closer Look at Rehearsal-Free Continual Learning

Continual learning describes a setting where machine learning models lea...
research
09/06/2023

Rethinking Momentum Knowledge Distillation in Online Continual Learning

Online Continual Learning (OCL) addresses the problem of training neural...
research
07/06/2020

Continual Learning in Human Activity Recognition: an Empirical Analysis of Regularization

Given the growing trend of continual learning techniques for deep neural...
research
03/22/2022

Meta-attention for ViT-backed Continual Learning

Continual learning is a longstanding research topic due to its crucial r...
research
06/04/2021

A Procedural World Generation Framework for Systematic Evaluation of Continual Learning

Several families of continual learning techniques have been proposed to ...

Please sign up or login with your details

Forgot password? Click here to reset