Doodle It Yourself: Class Incremental Learning by Drawing a Few Sketches

03/28/2022
by   Ayan Kumar Bhunia, et al.
6

The human visual system is remarkable in learning new visual concepts from just a few examples. This is precisely the goal behind few-shot class incremental learning (FSCIL), where the emphasis is additionally placed on ensuring the model does not suffer from "forgetting". In this paper, we push the boundary further for FSCIL by addressing two key questions that bottleneck its ubiquitous application (i) can the model learn from diverse modalities other than just photo (as humans do), and (ii) what if photos are not readily accessible (due to ethical and privacy constraints). Our key innovation lies in advocating the use of sketches as a new modality for class support. The product is a "Doodle It Yourself" (DIY) FSCIL framework where the users can freely sketch a few examples of a novel class for the model to learn to recognize photos of that class. For that, we present a framework that infuses (i) gradient consensus for domain invariant learning, (ii) knowledge distillation for preserving old class information, and (iii) graph attention networks for message passing between old and novel classes. We experimentally show that sketches are better class support than text in the context of FSCIL, echoing findings elsewhere in the sketching literature.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/11/2022

Memorizing Complementation Network for Few-Shot Class-Incremental Learning

Few-shot Class-Incremental Learning (FSCIL) aims at learning new concept...
research
10/12/2022

Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation

Class-incremental semantic segmentation (CISS) labels each pixel of an i...
research
08/07/2023

Heterogeneous Forgetting Compensation for Class-Incremental Learning

Class-incremental learning (CIL) has achieved remarkable successes in le...
research
03/06/2021

Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning

Few-shot class incremental learning (FSCIL) portrays the problem of lear...
research
03/27/2023

Towards Open Temporal Graph Neural Networks

Graph neural networks (GNNs) for temporal graphs have recently attracted...
research
07/22/2023

Revisiting Distillation for Continual Learning on Visual Question Localized-Answering in Robotic Surgery

The visual-question localized-answering (VQLA) system can serve as a kno...

Please sign up or login with your details

Forgot password? Click here to reset