CrescendoNet: A Simple Deep Convolutional Neural Network with Ensemble Behavior

10/30/2017
by   Xiang Zhang, et al.
0

We introduce a new deep convolutional neural network, CrescendoNet, by stacking simple building blocks without residual connections. Each Crescendo block contains independent convolution paths with increased depths. The numbers of convolution layers and parameters are only increased linearly in Crescendo blocks. In experiments, CrescendoNet with only 15 layers outperforms almost all networks without residual connections on benchmark datasets, CIFAR10, CIFAR100, and SVHN. Given sufficient amount of data as in SVHN dataset, CrescendoNet with 15 layers and 4.1M parameters can match the performance of DenseNet-BC with 250 layers and 15.3M parameters. CrescendoNet provides a new way to construct high performance deep convolutional neural networks without residual connections. Moreover, through investigating the behavior and performance of subnetworks in CrescendoNet, we note that the high performance of CrescendoNet may come from its implicit ensemble behavior, which differs from the FractalNet that is also a deep convolutional neural network without residual connections. Furthermore, the independence between paths in CrescendoNet allows us to introduce a new path-wise training procedure, which can reduce the memory needed for training.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/01/2018

Tandem Blocks in Deep Convolutional Neural Networks

Due to the success of residual networks (resnets) and related architectu...
research
05/24/2016

FractalNet: Ultra-Deep Neural Networks without Residuals

We introduce a design strategy for neural network macro-architecture bas...
research
01/25/2018

Effective Building Block Design for Deep Convolutional Neural Networks using Search

Deep learning has shown promising results on many machine learning tasks...
research
10/01/2019

NESTA: Hamming Weight Compression-Based Neural Proc. Engine

In this paper, we present NESTA, a specialized Neural engine that signif...
research
11/22/2017

BlockDrop: Dynamic Inference Paths in Residual Networks

Very deep convolutional neural networks offer excellent recognition resu...
research
05/24/2019

Robust learning with implicit residual networks

In this effort we propose a new deep architecture utilizing residual blo...

Please sign up or login with your details

Forgot password? Click here to reset