Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks

12/18/2015
by   Li Shen, et al.
0

Learning deeper convolutional neural networks becomes a tendency in recent years. However, many empirical evidences suggest that performance improvement cannot be gained by simply stacking more layers. In this paper, we consider the issue from an information theoretical perspective, and propose a novel method Relay Backpropagation, that encourages the propagation of effective information through the network in training stage. By virtue of the method, we achieved the first place in ILSVRC 2015 Scene Classification Challenge. Extensive experiments on two challenging large scale datasets demonstrate the effectiveness of our method is not restricted to a specific dataset or network architecture. Our models will be available to the research community later.

READ FULL TEXT
research
05/31/2022

A comparative study of back propagation and its alternatives on multilayer perceptrons

The de facto algorithm for training the back pass of a feedforward neura...
research
05/15/2023

Training Neural Networks without Backpropagation: A Deeper Dive into the Likelihood Ratio Method

Backpropagation (BP) is the most important gradient estimation method fo...
research
04/28/2020

Breast Cancer Histopathology Image Classification with Deep Convolutional Neural Networks

This work addresses the problem of intra-class classification of Breast ...
research
10/19/2021

Understanding Convolutional Neural Networks from Theoretical Perspective via Volterra Convolution

This study proposes a general and unified perspective of convolutional n...
research
01/23/2019

Decoupled Greedy Learning of CNNs

A commonly cited inefficiency of neural network training by back-propaga...
research
05/03/2018

Local Critic Training for Model-Parallel Learning of Deep Neural Networks

This paper proposes a novel approach to train deep neural networks in a ...
research
02/26/2019

Recurrent Convolution for Compact and Cost-Adjustable Neural Networks: An Empirical Study

Recurrent convolution (RC) shares the same convolutional kernels and unr...

Please sign up or login with your details

Forgot password? Click here to reset