DeepAI AI Chat
Log In Sign Up

DEPARA: Deep Attribution Graph for Deep Knowledge Transferability

by   Jie Song, et al.

Exploring the intrinsic interconnections between the knowledge encoded in PRe-trained Deep Neural Networks (PR-DNNs) of heterogeneous tasks sheds light on their mutual transferability, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter. In this paper, we propose the DEeP Attribution gRAph (DEPARA) to investigate the transferability of knowledge learned from PR-DNNs. In DEPARA, nodes correspond to the inputs and are represented by their vectorized attribution maps with regards to the outputs of the PR-DNN. Edges denote the relatedness between inputs and are measured by the similarity of their features extracted from the PR-DNN. The knowledge transferability of two PR-DNNs is measured by the similarity of their corresponding DEPARAs. We apply DEPARA to two important yet under-studied problems in transfer learning: pre-trained model selection and layer selection. Extensive experiments are conducted to demonstrate the effectiveness and superiority of the proposed method in solving both these problems. Code, data and models reproducing the results in this paper are available at <>.


page 4

page 6

page 8


Deep Model Transferability from Attribution Maps

Exploring the transferability between heterogeneous tasks sheds light on...

Rethinking Two Consensuses of the Transferability in Deep Learning

Deep transfer learning (DTL) has formed a long-term quest toward enablin...

Improving Adversarial Transferability via Neuron Attribution-Based Attacks

Deep neural networks (DNNs) are known to be vulnerable to adversarial ex...

Ranking Neural Checkpoints

This paper is concerned with ranking many pre-trained deep neural networ...

Frustratingly Easy Transferability Estimation

Transferability estimation has been an essential tool in selecting a pre...

Directing DNNs Attention for Facial Attribution Classification using Gradient-weighted Class Activation Mapping

Deep neural networks (DNNs) have a high accuracy on image classification...