Adversarial Training Helps Transfer Learning via Better Representations

06/18/2021
by   Zhun Deng, et al.
0

Transfer learning aims to leverage models pre-trained on source data to efficiently adapt to target setting, where only limited data are available for model fine-tuning. Recent works empirically demonstrate that adversarial training in the source data can improve the ability of models to transfer to new domains. However, why this happens is not known. In this paper, we provide a theoretical model to rigorously analyze how adversarial training helps transfer learning. We show that adversarial training in the source data generates provably better representations, so fine-tuning on top of this representation leads to a more accurate predictor of the target data. We further demonstrate both theoretically and empirically that semi-supervised learning in the source data can also improve transfer learning by similarly improving the representation. Moreover, performing adversarial training on top of semi-supervised learning can further improve transferability, suggesting that the two approaches have complementary benefits on representations. We support our theories with experiments on popular data sets and deep learning architectures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/03/2020

Meta-learning Transferable Representations with a Single Target Domain

Recent works found that fine-tuning and joint training—two popular appro...
research
07/14/2020

Adversarially-Trained Deep Nets Transfer Better

Transfer learning has emerged as a powerful methodology for adapting pre...
research
02/08/2021

Improving filling level classification with adversarial training

We investigate the problem of classifying - from a single image - the le...
research
06/29/2020

Adversarial Multi-Source Transfer Learning in Healthcare: Application to Glucose Prediction for Diabetic People

Deep learning has yet to revolutionize general practices in healthcare, ...
research
07/22/2020

Adversarial Training Reduces Information and Improves Transferability

Recent results show that features of adversarially trained networks for ...
research
10/30/2019

Is Supervised Learning With Adversarial Features Provably Better Than Sole Supervision?

Generative Adversarial Networks (GAN) have shown promising results on a ...
research
11/21/2022

Novel transfer learning schemes based on Siamese networks and synthetic data

Transfer learning schemes based on deep networks which have been trained...

Please sign up or login with your details

Forgot password? Click here to reset