DeepAI
Log In Sign Up

Convolutional Neural Network Simplification with Progressive Retraining

01/12/2021
by   D. Osaku, et al.
0

Kernel pruning methods have been proposed to speed up, simplify, and improve explanation of convolutional neural network (CNN) models. However, the effectiveness of a simplified model is often below the original one. In this letter, we present new methods based on objective and subjective relevance criteria for kernel elimination in a layer-by-layer fashion. During the process, a CNN model is retrained only when the current layer is entirely simplified, by adjusting the weights from the next layer to the first one and preserving weights of subsequent layers not involved in the process. We call this strategy progressive retraining, differently from kernel pruning methods that usually retrain the entire model after each simplification action – e.g., the elimination of one or a few kernels. Our subjective relevance criterion exploits the ability of humans in recognizing visual patterns and improves the designer's understanding of the simplification process. The combination of suitable relevance criteria and progressive retraining shows that our methods can increase effectiveness with considerable model simplification. We also demonstrate that our methods can provide better results than two popular ones and another one from the state-of-the-art using four challenging image datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/20/2017

Structured Probabilistic Pruning for Convolutional Neural Network Acceleration

Although deep Convolutional Neural Network (CNN) has shown better perfor...
12/18/2019

Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning

The success of convolutional neural networks (CNNs) in various applicati...
10/30/2016

Compact Deep Convolutional Neural Networks With Coarse Pruning

The learning capability of a neural network improves with increasing dep...
08/16/2018

Deep Convolutional Networks as shallow Gaussian Processes

We show that the output of a (residual) convolutional neural network (CN...
05/08/2020

Pruning Algorithms to Accelerate Convolutional Neural Networks for Edge Applications: A Survey

With the general trend of increasing Convolutional Neural Network (CNN) ...
08/06/2021

Basis Scaling and Double Pruning for Efficient Transfer Learning

Transfer learning allows the reuse of deep learning features on new data...
03/19/2021

Prediction of progressive lens performance from neural network simulations

Purpose: The purpose of this study is to present a framework to predict ...