Randomized Transferable Machine

11/27/2020
by   Pengfei Wei, et al.
0

Feature-based transfer is one of the most effective methodologies for transfer learning. Existing studies usually assume that the learned new feature representation is truly domain-invariant, and thus directly train a transfer model ℳ on source domain. In this paper, we consider a more realistic scenario where the new feature representation is suboptimal and small divergence still exists across domains. We propose a new learning strategy with a transfer model called Randomized Transferable Machine (RTM). More specifically, we work on source data with the new feature representation learned from existing feature-based transfer methods. The key idea is to enlarge source training data populations by randomly corrupting source data using some noises, and then train a transfer model ℳ that performs well on all the corrupted source data populations. In principle, the more corruptions are made, the higher the probability of the target data can be covered by the constructed source populations, and thus better transfer performance can be achieved by ℳ. An ideal case is with infinite corruptions, which however is infeasible in reality. We develop a marginalized solution with linear regression model and dropout noise. With a marginalization trick, we can train an RTM that is equivalently to training using infinite source noisy populations without truly conducting any corruption. More importantly, such an RTM has a closed-form solution, which enables very fast and efficient training. Extensive experiments on various real-world transfer tasks show that RTM is a promising transfer model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/31/2017

Transfer Learning with Label Noise

Transfer learning aims to improve learning in the target domain with lim...
research
10/16/2020

PrivNet: Safeguarding Private Attributes in Transfer Learning for Recommendation

Transfer learning is an effective technique to improve a target recommen...
research
10/23/2022

Robust angle-based transfer learning in high dimensions

Transfer learning aims to improve the performance of a target model by l...
research
05/25/2023

Representation Transfer Learning via Multiple Pre-trained models for Linear Regression

In this paper, we consider the problem of learning a linear regression m...
research
06/29/2020

Adversarial Multi-Source Transfer Learning in Healthcare: Application to Glucose Prediction for Diabetic People

Deep learning has yet to revolutionize general practices in healthcare, ...
research
04/29/2022

Fix the Noise: Disentangling Source Feature for Transfer Learning of StyleGAN

Transfer learning of StyleGAN has recently shown great potential to solv...
research
08/06/2014

Scalable Greedy Algorithms for Transfer Learning

In this paper we consider the binary transfer learning problem, focusing...

Please sign up or login with your details

Forgot password? Click here to reset