DeepAI AI Chat
Log In Sign Up

How Well Do Self-Supervised Methods Perform in Cross-Domain Few-Shot Learning?

02/18/2022
by   Yiyi Zhang, et al.
Microsoft
3

Cross-domain few-shot learning (CDFSL) remains a largely unsolved problem in the area of computer vision, while self-supervised learning presents a promising solution. Both learning methods attempt to alleviate the dependency of deep networks on the requirement of large-scale labeled data. Although self-supervised methods have recently advanced dramatically, their utility on CDFSL is relatively unexplored. In this paper, we investigate the role of self-supervised representation learning in the context of CDFSL via a thorough evaluation of existing methods. It comes as a surprise that even with shallow architectures or small training datasets, self-supervised methods can perform favorably compared to the existing SOTA methods. Nevertheless, no single self-supervised approach dominates all datasets indicating that existing self-supervised methods are not universally applicable. In addition, we find that representations extracted from self-supervised methods exhibit stronger robustness than the supervised method. Intriguingly, whether self-supervised representations perform well on the source domain has little correlation with their applicability on the target domain. As part of our study, we conduct an objective measurement of the performance for six kinds of representative classifiers. The results suggest Prototypical Classifier as the standard evaluation recipe for CDFSL.

READ FULL TEXT
01/19/2021

Cross-domain few-shot learning with unlabelled data

Few shot learning aims to solve the data scarcity problem. If there is a...
06/17/2019

Boosting Supervision with Self-Supervision for Few-shot Learning

We present a technique to improve the transferability of deep representa...
11/18/2022

Weighted Ensemble Self-Supervised Learning

Ensembling has proven to be a powerful technique for boosting model perf...
10/21/2021

Self-Supervised Visual Representation Learning Using Lightweight Architectures

In self-supervised learning, a model is trained to solve a pretext task,...
05/26/2023

Unsupervised Embedding Quality Evaluation

Unsupervised learning has recently significantly gained in popularity, e...
03/09/2023

Robust Holographic mmWave Beamforming by Self-Supervised Hybrid Deep Learning

Beamforming with large-scale antenna arrays has been widely used in rece...