P2L: Predicting Transfer Learning for Images and Semantic Relations

Transfer learning enhances learning across tasks, by leveraging previously learned representations -- if they are properly chosen. We describe an efficient method to accurately estimate the appropriateness of a previously trained model for use in a new learning task. We use this measure, which we call "Predict To Learn" ("P2L"), in the two very different domains of images and semantic relations, where it predicts, from a set of "source" models, the one model most likely to produce effective transfer for training a given "target" model. We validate our approach thoroughly, by assembling a collection of candidate source models, then fine-tuning each candidate to perform each of a collection of target tasks, and finally measuring how well transfer has been enhanced. Across 95 tasks within multiple domains (images classification and semantic relations), the P2L approach was able to select the best transfer learning model on average, while the heuristic of choosing model trained with the largest data set selected the best model in only 55 cases. These results suggest that P2L captures important information in common between source and target tasks, and that this shared informational structure contributes to successful transfer learning more than simple data size.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/11/2017

Investigation on How Data Volume Affects Transfer Learning Performances in Business Applications

Transfer Learning helps to build a system to recognize and apply knowled...
research
12/11/2017

Investigating the Impact of Data Volume and Domain Similarity on Transfer Learning Applications

Transfer Learning helps to build a system to recognize and apply knowled...
research
06/11/2020

Blissful Ignorance: Anti-Transfer Learning for Task Invariance

We introduce the novel concept of anti-transfer learning for neural netw...
research
05/31/2018

Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

Deep Reinforcement Learning has managed to achieve state-of-the-art resu...
research
11/10/2020

Predicting Water Temperature Dynamics of Unmonitored Lakes with Meta Transfer Learning

Most environmental data come from a minority of well-observed sites. An ...
research
12/10/2019

Transfer Learning-Based Outdoor Position Recovery with Telco Data

Telecommunication (Telco) outdoor position recovery aims to localize out...
research
10/24/2022

Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning

Knowing the cause of kidney stone formation is crucial to establish trea...

Please sign up or login with your details

Forgot password? Click here to reset