DeepAI
Log In Sign Up

Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data

06/14/2021
by   Ashraful Islam, et al.
0

Most existing works in few-shot learning rely on meta-learning the network on a large base dataset which is typically from the same domain as the target dataset. We tackle the problem of cross-domain few-shot learning where there is a large shift between the base and target domain. The problem of cross-domain few-shot recognition with unlabeled target data is largely unaddressed in the literature. STARTUP was the first method that tackles this problem using self-training. However, it uses a fixed teacher pretrained on a labeled base dataset to create soft labels for the unlabeled target samples. As the base dataset and unlabeled dataset are from different domains, projecting the target images in the class-domain of the base dataset with a fixed pretrained model might be sub-optimal. We propose a simple dynamic distillation-based approach to facilitate unlabeled images from the novel/base dataset. We impose consistency regularization by calculating predictions from the weakly-augmented versions of the unlabeled images from a teacher network and matching it with the strongly augmented versions of the same images from a student network. The parameters of the teacher network are updated as exponential moving average of the parameters of the student network. We show that the proposed network learns representation that can be easily adapted to the target domain even though it has not been trained with target-specific classes during the pretraining phase. Our model outperforms the current state-of-the art method by 4.4 and 3.6 competitive performance on traditional in-domain few-shot learning task. Our code will be available at: https://github.com/asrafulashiq/dynamic-cdfsl.

READ FULL TEXT
08/23/2022

Adversarial Feature Augmentation for Cross-domain Few-shot Classification

Existing methods based on meta-learning predict novel-class labels for (...
10/11/2022

ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning

Recently, Cross-Domain Few-Shot Learning (CD-FSL) which aims at addressi...
10/13/2020

Cross-Domain Few-Shot Learning by Representation Fusion

In order to quickly adapt to new data, few-shot learning aims at learnin...
07/26/2021

Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data

A recent study finds that existing few-shot learning methods, trained on...
10/08/2022

Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts

In this paper, we tackle the problem of domain shift. Most existing meth...
07/16/2022

Learn-to-Decompose: Cascaded Decomposition Network for Cross-Domain Few-Shot Facial Expression Recognition

Most existing compound facial expression recognition (FER) methods rely ...
11/03/2022

Robust Few-shot Learning Without Using any Adversarial Samples

The high cost of acquiring and annotating samples has made the `few-shot...