DeepAI AI Chat
Log In Sign Up

Ranking Distance Calibration for Cross-Domain Few-Shot Learning

12/01/2021
by   Pan Li, et al.
Queen Mary University of London
Tencent
FUDAN University
0

Recent progress in few-shot learning promotes a more realistic cross-domain setting, where the source and target datasets are from different domains. Due to the domain gap and disjoint label spaces between source and target datasets, their shared knowledge is extremely limited. This encourages us to explore more information in the target domain rather than to overly elaborate training strategies on the source domain as in many existing methods. Hence, we start from a generic representation pre-trained by a cross-entropy loss and a conventional distance-based classifier, along with an image retrieval view, to employ a re-ranking process for calibrating a target distance matrix by discovering the reciprocal k-nearest neighbours within the task. Assuming the pre-trained representation is biased towards the source, we construct a non-linear subspace to minimise task-irrelevant features therewithin while keep more transferrable discriminative information by a hyperbolic tangent transformation. The calibrated distance in this target-aware non-linear subspace is complementary to that in the pre-trained representation. To impose such distance calibration information onto the pre-trained representation, a Kullback-Leibler divergence loss is employed to gradually guide the model towards the calibrated distance-based distribution. Extensive evaluations on eight target domains show that this target ranking calibration process can improve conventional distance-based classifiers in few-shot learning.

READ FULL TEXT
05/11/2022

ReFine: Re-randomization before Fine-tuning for Cross-domain Few-shot Learning

Cross-domain few-shot learning (CD-FSL), where there are few target samp...
12/05/2022

Cross-Domain Few-Shot Relation Extraction via Representation Learning and Domain Adaptation

Cross-domain few-shot relation extraction poses a great challenge for th...
10/15/2020

Self-training for Few-shot Transfer Across Extreme Task Differences

All few-shot learning techniques must be pre-trained on a large, labeled...
09/03/2021

Self-Taught Cross-Domain Few-Shot Learning with Weakly Supervised Object Localization and Task-Decomposition

The domain shift between the source and target domain is the main challe...
03/06/2022

Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment

Training a generative adversarial network (GAN) with limited data has be...
03/15/2023

Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey

Deep learning has been highly successful in computer vision with large a...
12/23/2019

How to Pick the Best Source Data? Measuring Transferability for Heterogeneous Domains

Given a set of source data with pre-trained classification models, how c...