DeepAI AI Chat
Log In Sign Up

Wide Residual Networks

05/23/2016
by   Sergey Zagoruyko, et al.
0

Deep residual networks were shown to be able to scale up to thousands of layers and still have improving performance. However, each fraction of a percent of improved accuracy costs nearly doubling the number of layers, and so training very deep residual networks has a problem of diminishing feature reuse, which makes these networks very slow to train. To tackle these problems, in this paper we conduct a detailed experimental study on the architecture of ResNet blocks, based on which we propose a novel architecture where we decrease depth and increase width of residual networks. We call the resulting network structures wide residual networks (WRNs) and show that these are far superior over their commonly used thin and very deep counterparts. For example, we demonstrate that even a simple 16-layer-deep wide residual network outperforms in accuracy and efficiency all previous deep residual networks, including thousand-layer-deep networks, achieving new state-of-the-art results on CIFAR, SVHN, COCO, and significant improvements on ImageNet. Our code and models are available at https://github.com/szagoruyko/wide-residual-networks

READ FULL TEXT

page 1

page 2

page 3

page 4

04/10/2020

Improved Residual Networks for Image and Video Recognition

Residual networks (ResNets) represent a powerful type of convolutional n...
02/28/2017

ShaResNet: reducing residual network parameter number by sharing weights

Deep Residual Networks have reached the state of the art in many image p...
08/09/2016

Residual Networks of Residual Networks: Multilevel Residual Networks

A residual-networks family with hundreds or even thousands of layers dom...
11/30/2016

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

The trend towards increasingly deep neural networks has been driven by a...
10/10/2016

Deep Pyramidal Residual Networks

Deep convolutional neural networks (DCNNs) have shown remarkable perform...
11/23/2020

Scaling Wide Residual Networks for Panoptic Segmentation

The Wide Residual Networks (Wide-ResNets), a shallow but wide model vari...
04/03/2017

Truncating Wide Networks using Binary Tree Architectures

Recent study shows that a wide deep network can obtain accuracy comparab...

Code Repositories

wide-residual-networks

3.8% and 18.3% on CIFAR-10 and CIFAR-100


view repo

WideResNet-pytorch

Wide Residual Networks (WideResNets) in PyTorch


view repo

wide_residual_nets_caffe

training wide residual networks in caffe


view repo

sharesnet

ShaResNet: reducing residual network parameter number by sharing weights


view repo

resnet-cifar10

ResNet for Cifar10


view repo