GradMix: Multi-source Transfer across Domains and Tasks

02/09/2020
by   Junnan Li, et al.
0

The computer vision community is witnessing an unprecedented rate of new tasks being proposed and addressed, thanks to the deep convolutional networks' capability to find complex mappings from X to Y. The advent of each task often accompanies the release of a large-scale annotated dataset, for supervised training of deep network. However, it is expensive and time-consuming to manually label sufficient amount of training data. Therefore, it is important to develop algorithms that can leverage off-the-shelf labeled dataset to learn useful knowledge for the target task. While previous works mostly focus on transfer learning from a single source, we study multi-source transfer across domains and tasks (MS-DTT), in a semi-supervised setting. We propose GradMix, a model-agnostic method applicable to any model trained with gradient-based learning rule, to transfer knowledge via gradient descent by weighting and mixing the gradients from all sources during training. GradMix follows a meta-learning objective, which assigns layer-wise weights to the source gradients, such that the combined gradient follows the direction that minimize the loss for a small set of samples from the target dataset. In addition, we propose to adaptively adjust the learning rate for each mini-batch based on its importance to the target task, and a pseudo-labeling method to leverage the unlabeled samples in the target domain. We conduct MS-DTT experiments on two tasks: digit recognition and action recognition, and demonstrate the advantageous performance of the proposed method against multiple baselines.

READ FULL TEXT
research
04/27/2022

Transfer Learning with Pre-trained Conditional Generative Models

Transfer learning is crucial in training deep neural networks on new tar...
research
04/24/2023

Distilling from Similar Tasks for Transfer Learning on a Budget

We address the challenge of getting efficient yet accurate recognition s...
research
06/11/2023

An information-Theoretic Approach to Semi-supervised Transfer Learning

Transfer learning is a valuable tool in deep learning as it allows propa...
research
11/30/2017

Label Efficient Learning of Transferable Representations across Domains and Tasks

We propose a framework that learns a representation transferable across ...
research
09/15/2014

Transfer Learning for Video Recognition with Scarce Training Data for Deep Convolutional Neural Network

Unconstrained video recognition and Deep Convolution Network (DCN) are t...
research
04/27/2022

Adaptable Text Matching via Meta-Weight Regulator

Neural text matching models have been used in a range of applications su...

Please sign up or login with your details

Forgot password? Click here to reset