Convolutional Residual Memory Networks

06/16/2016
by   Joel Moniz, et al.
0

Very deep convolutional neural networks (CNNs) yield state of the art results on a wide variety of visual recognition problems. A number of state of the the art methods for image recognition are based on networks with well over 100 layers and the performance vs. depth trend is moving towards networks in excess of 1000 layers. In such extremely deep architectures the vanishing or exploding gradient problem becomes a key issue. Recent evidence also indicates that convolutional networks could benefit from an interface to explicitly constructed memory mechanisms interacting with a CNN feature processing hierarchy. Correspondingly, we propose and evaluate a memory mechanism enhanced convolutional neural network architecture based on augmenting convolutional residual networks with a long short term memory mechanism. We refer to this as a convolutional residual memory network. To the best of our knowledge this approach can yield state of the art performance on the CIFAR-100 benchmark and compares well with other state of the art techniques on the CIFAR-10 and SVHN benchmarks. This is achieved using networks with more breadth, much less depth and much less overall computation relative to comparable deep ResNets without the memory mechanism. Our experiments and analysis explore the importance of the memory mechanism, network depth, breadth, and predictive performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/31/2015

Beyond Short Snippets: Deep Networks for Video Classification

Convolutional neural networks (CNNs) have been extensively applied for i...
research
11/25/2017

Gradually Updated Neural Networks for Large-Scale Image Recognition

We present a simple yet effective neural network architecture for image ...
research
11/06/2016

The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs

The depth is one of the key factors behind the great success of convolut...
research
08/30/2018

Total Recall: Understanding Traffic Signs using Deep Hierarchical Convolutional Neural Networks

Recognizing Traffic Signs using intelligent systems can drastically redu...
research
11/27/2017

Context-modulation of hippocampal dynamics and deep convolutional networks

Complex architectures of biological neural circuits, such as parallel pr...
research
10/18/2018

S-Net: A Scalable Convolutional Neural Network for JPEG Compression Artifact Reduction

Recent studies have used deep residual convolutional neural networks (CN...

Please sign up or login with your details

Forgot password? Click here to reset