Deep Convolutional Neural Networks with Merge-and-Run Mappings

11/23/2016
by   Liming Zhao, et al.
0

A deep residual network, built by stacking a sequence of residual blocks, is easy to train, because identity mappings skip residual branches and thus improve information flow. To further reduce the training difficulty, we present a simple network architecture, deep merge-and-run neural networks. The novelty lies in a modularized building block, merge-and-run block, which assembles residual branches in parallel through a merge-and-run mapping: Average the inputs of these residual branches (Merge), and add the average to the output of each residual branch as the input of the subsequent residual branch (Run), respectively. We show that the merge-and-run mapping is a linear idempotent function in which the transformation matrix is idempotent, and thus improves information flow, making training easy. In comparison to residual networks, our networks enjoy compelling advantages: they contain much shorter paths, and the width, i.e., the number of channels, is increased. We evaluate the performance on the standard recognition tasks. Our approach demonstrates consistent improvements over ResNets with the comparable setup, and achieves competitive results (e.g., 3.57% testing error on CIFAR-10, 19.00% on CIFAR-100, 1.51% on SVHN).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/24/2018

Competitive Inner-Imaging Squeeze and Excitation for Residual Network

Residual Network make the very deep convolutional architecture works wel...
research
03/16/2016

Identity Mappings in Deep Residual Networks

Deep residual networks have emerged as a family of extremely deep archit...
research
08/09/2016

Residual Networks of Residual Networks: Multilevel Residual Networks

A residual-networks family with hundreds or even thousands of layers dom...
research
06/26/2019

On the performance of residual block design alternatives in convolutional neural networks for end-to-end audio classification

Residual learning is a recently proposed learning framework to facilitat...
research
10/15/2018

Channel Splitting Network for Single MR Image Super-Resolution

High resolution magnetic resonance (MR) imaging is desirable in many cli...
research
02/11/2019

On Residual Networks Learning a Perturbation from Identity

The purpose of this work is to test and study the hypothesis that residu...
research
05/21/2017

Shake-Shake regularization

The method introduced in this paper aims at helping deep learning practi...

Please sign up or login with your details

Forgot password? Click here to reset