Self-Reorganizing and Rejuvenating CNNs for Increasing Model Capacity Utilization

02/13/2021
by   Wissam J. Baddar, et al.
0

In this paper, we propose self-reorganizing and rejuvenating convolutional neural networks; a biologically inspired method for improving the computational resource utilization of neural networks. The proposed method utilizes the channel activations of a convolution layer in order to reorganize that layers parameters. The reorganized parameters are clustered to avoid parameter redundancies. As such, redundant neurons with similar activations are merged leaving room for the remaining parameters to rejuvenate. The rejuvenated parameters learn different features to supplement those learned by the reorganized surviving parameters. As a result, the network capacity utilization increases improving the baseline network performance without any changes to the network structure. The proposed method can be applied to various network architectures during the training stage, or applied to a pre-trained model improving its performance. Experimental results showed that the proposed method is model-agnostic and can be applied to any backbone architecture increasing its performance due to the elevated utilization of the network capacity.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 7

05/11/2019

Training CNNs with Selective Allocation of Channels

Recent progress in deep convolutional neural networks (CNNs) have enable...
12/02/2018

Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization

In this paper, we study the problem of improving computational resource ...
07/20/2020

Improving Memory Utilization in Convolutional Neural Network Accelerators

While the accuracy of convolutional neural networks has achieved vast im...
07/31/2021

Greedy Network Enlarging

Recent studies on deep convolutional neural networks present a simple pa...
07/31/2017

An Effective Training Method For Deep Convolutional Neural Network

In this paper, we propose the nonlinearity generation method to speed up...
01/14/2019

Bayesian Learning of Neural Network Architectures

In this paper we propose a Bayesian method for estimating architectural ...
05/28/2020

Exploiting Non-Linear Redundancy for Neural Model Compression

Deploying deep learning models, comprising of non-linear combination of ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.