Duality Diagram Similarity: a generic framework for initialization selection in task transfer learning

08/05/2020
by   Kshitij Dwivedi, et al.
4

In this paper, we tackle an open research question in transfer learning, which is selecting a model initialization to achieve high performance on a new task, given several pre-trained models. We propose a new highly efficient and accurate approach based on duality diagram similarity (DDS) between deep neural networks (DNNs). DDS is a generic framework to represent and compare data of different feature dimensions. We validate our approach on the Taskonomy dataset by measuring the correspondence between actual transfer learning performance rankings on 17 taskonomy tasks and predicted rankings. Computing DDS based ranking for 17×17 transfers requires less than 2 minutes and shows a high correlation (0.86) with actual transfer learning rankings, outperforming state-of-the-art methods by a large margin (10%) on the Taskonomy benchmark. We also demonstrate the robustness of our model selection approach to a new task, namely Pascal VOC semantic segmentation. Additionally, we show that our method can be applied to select the best layer locations within a DNN for transfer learning on 2D, 3D and semantic tasks on NYUv2 and Pascal VOC datasets.

READ FULL TEXT

page 28

page 29

research
04/26/2019

Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learning

Transfer learning is widely used in deep neural network models when ther...
research
12/26/2020

Ranking and Rejecting of Pre-Trained Deep Neural Networks in Transfer Learning based on Separation Index

Automated ranking of pre-trained Deep Neural Networks (DNNs) reduces the...
research
02/22/2021

LogME: Practical Assessment of Pre-trained Models for Transfer Learning

This paper studies task adaptive pre-trained model selection, an underex...
research
11/23/2020

Ranking Neural Checkpoints

This paper is concerned with ranking many pre-trained deep neural networ...
research
06/19/2018

Transfer Learning with Human Corneal Tissues: An Analysis of Optimal Cut-Off Layer

Transfer learning is a powerful tool to adapt trained neural networks to...
research
04/21/2023

How good are variational autoencoders at transfer learning?

Variational autoencoders (VAEs) are used for transfer learning across va...
research
04/28/2022

Model Selection, Adaptation, and Combination for Deep Transfer Learning through Neural Networks in Renewable Energies

There is recent interest in using model hubs, a collection of pre-traine...

Please sign up or login with your details

Forgot password? Click here to reset