Uncertainty-Aware Distillation for Semi-Supervised Few-Shot Class-Incremental Learning

01/24/2023
by   Yawen Cui, et al.
0

Given a model well-trained with a large-scale base dataset, Few-Shot Class-Incremental Learning (FSCIL) aims at incrementally learning novel classes from a few labeled samples by avoiding overfitting, without catastrophically forgetting all encountered classes previously. Currently, semi-supervised learning technique that harnesses freely-available unlabeled data to compensate for limited labeled data can boost the performance in numerous vision tasks, which heuristically can be applied to tackle issues in FSCIL, i.e., the Semi-supervised FSCIL (Semi-FSCIL). So far, very limited work focuses on the Semi-FSCIL task, leaving the adaptability issue of semi-supervised learning to the FSCIL task unresolved. In this paper, we focus on this adaptability issue and present a simple yet efficient Semi-FSCIL framework named Uncertainty-aware Distillation with Class-Equilibrium (UaD-CE), encompassing two modules UaD and CE. Specifically, when incorporating unlabeled data into each incremental session, we introduce the CE module that employs a class-balanced self-training to avoid the gradual dominance of easy-to-classified classes on pseudo-label generation. To distill reliable knowledge from the reference model, we further implement the UaD module that combines uncertainty-guided knowledge refinement with adaptive distillation. Comprehensive experiments on three benchmark datasets demonstrate that our method can boost the adaptability of unlabeled data with the semi-supervised learning technique in FSCIL tasks.

READ FULL TEXT
research
10/21/2021

A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning

Few-shot learning (FSL) aims to learn models that generalize to novel cl...
research
07/20/2022

Rethinking Few-Shot Class-Incremental Learning with Open-Set Hypothesis in Hyperbolic Geometry

Few-Shot Class-Incremental Learning (FSCIL) aims at incrementally learni...
research
12/15/2021

Taming Overconfident Prediction on Unlabeled Data from Hindsight

Minimizing prediction uncertainty on unlabeled data is a key factor to a...
research
04/19/2022

Incorporating Semi-Supervised and Positive-Unlabeled Learning for Boosting Full Reference Image Quality Assessment

Full-reference (FR) image quality assessment (IQA) evaluates the visual ...
research
05/31/2022

Semi-Supervised Cross-Silo Advertising with Partial Knowledge Transfer

As an emerging secure learning paradigm in leveraging cross-agency priva...
research
06/29/2021

GuidedMix-Net: Learning to Improve Pseudo Masks Using Labeled Images as Reference

Semi-supervised learning is a challenging problem which aims to construc...
research
08/23/2023

Semi-Supervised Learning via Weight-aware Distillation under Class Distribution Mismatch

Semi-Supervised Learning (SSL) under class distribution mismatch aims to...

Please sign up or login with your details

Forgot password? Click here to reset