DeepAI AI Chat
Log In Sign Up

Batch-normalized Maxout Network in Network

11/09/2015
by   Jia-Ren Chang, et al.
National Chiao Tung University
0

This paper reports a novel deep architecture referred to as Maxout network In Network (MIN), which can enhance model discriminability and facilitate the process of information abstraction within the receptive field. The proposed network adopts the framework of the recently developed Network In Network structure, which slides a universal approximator, multilayer perceptron (MLP) with rectifier units, to exact features. Instead of MLP, we employ maxout MLP to learn a variety of piecewise linear activation functions and to mediate the problem of vanishing gradients that can occur when using rectifier units. Moreover, batch normalization is applied to reduce the saturation of maxout units by pre-conditioning the model and dropout is applied to prevent overfitting. Finally, average pooling is used in all pooling layers to regularize maxout MLP in order to facilitate information abstraction in every receptive field while tolerating the change of object position. Because average pooling preserves all features in the local patch, the proposed MIN model can enforce the suppression of irrelevant information during training. Our experiments demonstrated the state-of-the-art classification performance when the MIN model was applied to MNIST, CIFAR-10, and CIFAR-100 datasets and comparable performance for SVHN dataset.

READ FULL TEXT

page 7

page 8

12/16/2013

Network In Network

We propose a novel deep network structure called "Network In Network" (N...
08/03/2015

On the Importance of Normalisation Layers in Deep Learning with Piecewise Linear Activation Units

Deep feedforward neural networks with piecewise linear activations are c...
10/07/2018

Hartley Spectral Pooling for Deep Learning

In most convolution neural networks (CNNs), downsampling hidden layers i...
06/23/2020

Density-embedding layers: a general framework for adaptive receptive fields

The effectiveness and performance of artificial neural networks, particu...
04/11/2018

Detail-Preserving Pooling in Deep Networks

Most convolutional neural networks use some method for gradually downsca...
12/20/2013

Improving Deep Neural Networks with Probabilistic Maxout Units

We present a probabilistic variant of the recently introduced maxout uni...
11/09/2019

XceptionTime: A Novel Deep Architecture based on Depthwise Separable Convolutions for Hand Gesture Classification

Capitalizing on the need for addressing the existing challenges associat...

Code Repositories

Batch_Normalized_Maxout_NIN

http://arxiv.org/abs/1511.02583


view repo

mcnMaxout

Maxout networks for MatConvNet


view repo