Semantic and Visual Similarities for Efficient Knowledge Transfer in CNN Training

by   Lucas Pascal, et al.

In recent years, representation learning approaches have disrupted many multimedia computing tasks. Among those approaches, deep convolutional neural networks (CNNs) have notably reached human level expertise on some constrained image classification tasks. Nonetheless, training CNNs from scratch for new task or simply new data turns out to be complex and time-consuming. Recently, transfer learning has emerged as an effective methodology for adapting pre-trained CNNs to new data and classes, by only retraining the last classification layer. This paper focuses on improving this process, in order to better transfer knowledge between CNN architectures for faster trainings in the case of fine tuning for image classification. This is achieved by combining and transfering supplementary weights, based on similarity considerations between source and target classes. The study includes a comparison between semantic and content-based similarities, and highlights increased initial performances and training speed, along with superior long term performances when limited training samples are available.


Convolutional Neural Networks for Histopathology Image Classification: Training vs. Using Pre-Trained Networks

We explore the problem of classification within a medical image data-set...

Optimal Transport for Deep Joint Transfer Learning

Training a Deep Neural Network (DNN) from scratch requires a large amoun...

On evaluating CNN representations for low resource medical image classification

Convolutional Neural Networks (CNNs) have revolutionized performances in...

Facilitated machine learning for image-based fruit quality assessment in developing countries

Automated image classification is a common task for supervised machine l...

Large Neural Networks Learning from Scratch with Very Few Data and without Regularization

Recent findings have shown that Neural Networks generalize also in over-...

A systematic review of transfer learning based approaches for diabetic retinopathy detection

Cases of diabetes and related diabetic retinopathy (DR) have been increa...

Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing

Deep convolutional neural network (DCNN) based supervised learning is a ...

Please sign up or login with your details

Forgot password? Click here to reset