N-fold Superposition: Improving Neural Networks by Reducing the Noise in Feature Maps

04/23/2018
by   Yang Liu, et al.
0

Considering the use of Fully Connected (FC) layer limits the performance of Convolutional Neural Networks (CNNs), this paper develops a method to improve the coupling between the convolution layer and the FC layer by reducing the noise in Feature Maps (FMs). Our approach is divided into three steps. Firstly, we separate all the FMs into n blocks equally. Then, the weighted summation of FMs at the same position in all blocks constitutes a new block of FMs. Finally, we replicate this new block into n copies and concatenate them as the input to the FC layer. This sharing of FMs could reduce the noise in them apparently and avert the impact by a particular FM on the specific part weight of hidden layers, hence preventing the network from overfitting to some extent. Using the Fermat Lemma, we prove that this method could make the global minima value range of the loss function wider, by which makes it easier for neural networks to converge and accelerates the convergence process. This method does not significantly increase the amounts of network parameters (only a few more coefficients added), and the experiments demonstrate that this method could increase the convergence speed and improve the classification performance of neural networks.

READ FULL TEXT
research
01/26/2019

Atrous Convolutional Neural Network (ACNN) for Biomedical Semantic Segmentation with Dimensionally Lossless Feature Maps

Deep Convolutional Neural Networks (DCNNs) are showing impressive perfor...
research
09/16/2020

Pooling Methods in Deep Neural Networks, a Review

Nowadays, Deep Neural Networks are among the main tools used in various ...
research
03/19/2020

Ensemble learning in CNN augmented with fully connected subnetworks

Convolutional Neural Networks (CNNs) have shown remarkable performance i...
research
06/20/2016

Learning Convolutional Neural Networks using Hybrid Orthogonal Projection and Estimation

Convolutional neural networks (CNNs) have yielded the excellent performa...
research
04/06/2019

LP-3DCNN: Unveiling Local Phase in 3D Convolutional Neural Networks

Traditional 3D Convolutional Neural Networks (CNNs) are computationally ...
research
12/16/2013

Network In Network

We propose a novel deep network structure called "Network In Network" (N...
research
09/03/2020

It's Hard for Neural Networks To Learn the Game of Life

Efforts to improve the learning abilities of neural networks have focuse...

Please sign up or login with your details

Forgot password? Click here to reset