-
SMILE: Self-Distilled MIxup for Efficient Transfer LEarning
To improve the performance of deep learning, mixup has been proposed to ...
03/25/2021 ∙ by Xingjian Li, et al. ∙
0
∙
share
read it
-
Deep Transfer Learning for WiFi Localization
This paper studies a WiFi indoor localisation technique based on using a...
03/08/2021 ∙ by Peizheng Li, et al. ∙
0
∙
share
read it
-
Explicit Inductive Bias for Transfer Learning with Convolutional Networks
In inductive transfer learning, fine-tuning pre-trained convolutional ne...
02/05/2018 ∙ by Xuhong Li, et al. ∙
0
∙
share
read it
-
Channel Scaling: A Scale-and-Select Approach for Transfer Learning
Transfer learning with pre-trained neural networks is a common strategy ...
03/22/2021 ∙ by Ken C. L. Wong, et al. ∙
0
∙
share
read it
-
Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-Identification
In person re-identification (ReID) task, because of its shortage of trai...
01/18/2019 ∙ by Youngmin Ro, et al. ∙
0
∙
share
read it
-
NetTailor: Tuning the Architecture, Not Just the Weights
Real-world applications of object recognition often require the solution...
06/29/2019 ∙ by Pedro Morgado, et al. ∙
0
∙
share
read it
-
AutoFCL: Automatically Tuning Fully Connected Layers for Transfer Learning
Deep Convolutional Neural Networks (CNN) have evolved as popular machine...
01/22/2020 ∙ by S. H. Shabbeer Basha, et al. ∙
0
∙
share
read it
RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr
07/07/2020 ∙ by Xingjian Li, et al. ∙
0
∙
share
Fine-tuning the deep convolution neural network(CNN) using a pre-trained model helps transfer knowledge learned from larger datasets to the target task. While the accuracy could be largely improved even when the training dataset is small, the transfer learning outcome is usually constrained by the pre-trained model with close CNN weights (Liu et al., 2019), as the backpropagation here brings smaller updates to deeper CNN layers. In this work, we propose RIFLE - a simple yet effective strategy that deepens backpropagation in transfer learning settings, through periodically Re-Initializing the Fully-connected LayEr with random scratch during the fine-tuning procedure. RIFLE brings meaningful updates to the weights of deep CNN layers and improves low-level feature learning, while the effects of randomization can be easily converged throughout the overall learning procedure. The experiments show that the use of RIFLE significantly improves deep transfer learning accuracy on a wide range of datasets, out-performing known tricks for the similar purpose, such as Dropout, DropConnect, StochasticDepth, Disturb Label and Cyclic Learning Rate, under the same settings with 0.5 ablation studies further indicate RIFLE brings meaningful updates to deep CNN layers with accuracy improved.
READ FULL TEXT
Comments
There are no comments yet.