Deep Collaborative Learning for Visual Recognition

03/03/2017
by   Yan Wang, et al.
0

Deep neural networks are playing an important role in state-of-the-art visual recognition. To represent high-level visual concepts, modern networks are equipped with large convolutional layers, which use a large number of filters and contribute significantly to model complexity. For example, more than half of the weights of AlexNet are stored in the first fully-connected layer (4,096 filters). We formulate the function of a convolutional layer as learning a large visual vocabulary, and propose an alternative way, namely Deep Collaborative Learning (DCL), to reduce the computational complexity. We replace a convolutional layer with a two-stage DCL module, in which we first construct a couple of smaller convolutional layers individually, and then fuse them at each spatial position to consider feature co-occurrence. In mathematics, DCL can be explained as an efficient way of learning compositional visual concepts, in which the vocabulary size increases exponentially while the model complexity only increases linearly. We evaluate DCL on a wide range of visual recognition tasks, including a series of multi-digit number classification datasets, and some generic image classification datasets such as SVHN, CIFAR and ILSVRC2012. We apply DCL to several state-of-the-art network structures, improving the recognition accuracy meanwhile reducing the number of parameters (16.82 in AlexNet).

READ FULL TEXT
research
06/05/2018

Perturbative Neural Networks

Convolutional neural networks are witnessing wide adoption in computer v...
research
02/28/2017

ShaResNet: reducing residual network parameter number by sharing weights

Deep Residual Networks have reached the state of the art in many image p...
research
10/03/2017

A concatenating framework of shortcut convolutional neural networks

It is well accepted that convolutional neural networks play an important...
research
10/21/2019

Separable Convolutional Eigen-Filters (SCEF): Building Efficient CNNs Using Redundancy Analysis

The high model complexity of deep learning algorithms enables remarkable...
research
09/25/2019

Gated Channel Transformation for Visual Recognition

In this work, we propose a generally applicable transformation unit for ...
research
08/26/2020

Visual Concept Reasoning Networks

A split-transform-merge strategy has been broadly used as an architectur...
research
09/07/2018

Accelerating Deep Neural Networks with Spatial Bottleneck Modules

This paper presents an efficient module named spatial bottleneck for acc...

Please sign up or login with your details

Forgot password? Click here to reset