Striving for Simplicity: The All Convolutional Net

by   Jost Tobias Springenberg, et al.
University of Freiburg

Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We find that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this finding -- and building on other recent work for finding simple network structures -- we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the "deconvolution approach" for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.


page 8

page 10

page 13

page 14


Robust Audio Event Recognition with 1-Max Pooling Convolutional Neural Networks

We present in this paper a simple, yet efficient convolutional neural ne...

Deep Epitomic Convolutional Neural Networks

Deep convolutional neural networks have recently proven extremely compet...

Multilevel Context Representation for Improving Object Recognition

In this work, we propose the combined usage of low- and high-level block...

Analyzing the Dependency of ConvNets on Spatial Information

Intuitively, image classification should profit from using spatial infor...

A Fully Trainable Network with RNN-based Pooling

Pooling is an important component in convolutional neural networks (CNNs...

A Deeper Look into Convolutions via Pruning

Convolutional neural networks (CNNs) are able to attain better visual re...

Parallel Convolutional Networks for Image Recognition via a Discriminator

In this paper, we introduce a simple but quite effective recognition fra...

Code Repositories


All Convolutional Network: ( implementation in Keras

view repo


Auto-optimizing a neural net (and its architecture) on the CIFAR-100 dataset. Could be easily transferred to another dataset or another classification task. Updated version here:

view repo


Implementation of Guided Backpropagation in Chainer (ChainerRL)

view repo


This repository contains the code for an all convolution CNN. Conventionally, CNN includes Maxpool and Fully connected layers. But in this network, they have been replaced by customised convolutional layers. PyTorch was used as the framework.

view repo



view repo

Please sign up or login with your details

Forgot password? Click here to reset