Normalized Convolutional Neural Network

05/11/2020
by   Dongsuk Kim, et al.
0

In this paper, we propose Normalized Convolutional Neural Network(NCNN). NCNN is more adaptive to a convolutional operator than other nomralizaiton methods. The normalized process is similar to a normalization methods, but NCNN is more adapative to sliced-inputs and corresponding the convolutional kernel. As NCNN is more adpative to sliced-inputs, NCNN can be targeted to micro-batch training. Normalizaing of NC is conducted during convolutional process. In short, NC process is not usual normalization and can not be realized in deep learning framework optimizing standard convolution process. Hence we named this method 'Normalized Convolution'. As a result, NC process has universal property which means NC can be applied to any AI tasks involving convolution neural layer . Since NC don't need other normalization layer, NCNN looks like convolutional version of Self Normalizing Network.(SNN). Among micro-batch trainings, NCNN outperforms other batch-independent normalization methods. NCNN archives these superiority by standardizing rows of im2col matrix of inputs, which theoretically smooths the gradient of loss. The code need to manipulate standard convolution neural networks step by step. The code is available : https://github.com/kimdongsuk1/ NormalizedCNN.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/25/2019

Weight Standardization

In this paper, we propose Weight Standardization (WS) to accelerate deep...
research
05/20/2022

Kernel Normalized Convolutional Networks

Existing deep convolutional neural network (CNN) architectures frequentl...
research
11/25/2019

Radius Adaptive Convolutional Neural Network

Convolutional neural network (CNN) is widely used in computer vision app...
research
02/27/2019

Regularity Normalization: Constraining Implicit Space with Minimum Description Length

Inspired by the adaptation phenomenon of biological neuronal firing rate...
research
10/16/2019

Root Mean Square Layer Normalization

Layer normalization (LayerNorm) has been successfully applied to various...
research
02/25/2020

Separating the Effects of Batch Normalization on CNN Training Speed and Stability Using Classical Adaptive Filter Theory

Batch Normalization (BatchNorm) is commonly used in Convolutional Neural...
research
12/27/2021

Self-normalized Classification of Parkinson's Disease DaTscan Images

Classifying SPECT images requires a preprocessing step which normalizes ...

Please sign up or login with your details

Forgot password? Click here to reset