-
Towards Making Deep Transfer Learning Never Hurt
Transfer learning have been frequently used to improve deep neural netwo...
read it
-
XMixup: Efficient Transfer Learning with Auxiliary Samples by Cross-domain Mixup
Transferring knowledge from large source datasets is an effective way to...
read it
-
Distance-Based Regularisation of Deep Networks for Fine-Tuning
We investigate approaches to regularisation during fine-tuning of deep n...
read it
-
SNN: Stacked Neural Networks
It has been proven that transfer learning provides an easy way to achiev...
read it
-
DAC: Data-free Automatic Acceleration of Convolutional Networks
Deploying a deep learning model on mobile/IoT devices is a challenging t...
read it
-
Tiny Transfer Learning: Towards Memory-Efficient On-Device Learning
We present Tiny-Transfer-Learning (TinyTL), an efficient on-device learn...
read it
-
Pseudo-task Regularization for ConvNet Transfer Learning
This paper is about regularizing deep convolutional networks (ConvNets) ...
read it
DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks
Transfer learning through fine-tuning a pre-trained neural network with an extremely large dataset, such as ImageNet, can significantly accelerate training while the accuracy is frequently bottlenecked by the limited dataset size of the new target task. To solve the problem, some regularization methods, constraining the outer layer weights of the target network using the starting point as references (SPAR), have been studied. In this paper, we propose a novel regularized transfer learning framework DELTA, namely DEep Learning Transfer using Feature Map with Attention. Instead of constraining the weights of neural network, DELTA aims to preserve the outer layer outputs of the target network. Specifically, in addition to minimizing the empirical loss, DELTA intends to align the outer layer outputs of two networks, through constraining a subset of feature maps that are precisely selected by attention that has been learned in an supervised learning manner. We evaluate DELTA with the state-of-the-art algorithms, including L2 and L2-SP. The experiment results show that our proposed method outperforms these baselines with higher accuracy for new tasks.
READ FULL TEXT
Comments
There are no comments yet.