Training CNNs with Selective Allocation of Channels

05/11/2019
by   Jongheon Jeong, et al.
4

Recent progress in deep convolutional neural networks (CNNs) have enabled a simple paradigm of architecture design: larger models typically achieve better accuracy. Due to this, in modern CNN architectures, it becomes more important to design models that generalize well under certain resource constraints, e.g. the number of parameters. In this paper, we propose a simple way to improve the capacity of any CNN model having large-scale features, without adding more parameters. In particular, we modify a standard convolutional layer to have a new functionality of channel-selectivity, so that the layer is trained to select important channels to re-distribute their parameters. Our experimental results under various CNN architectures and datasets demonstrate that the proposed new convolutional layer allows new optima that generalize better via efficient resource utilization, compared to the baseline.

READ FULL TEXT

page 7

page 14

research
06/11/2020

Multigrid-in-Channels Architectures for Wide Convolutional Neural Networks

We present a multigrid approach that combats the quadratic growth of the...
research
07/31/2021

Greedy Network Enlarging

Recent studies on deep convolutional neural networks present a simple pa...
research
02/13/2021

Self-Reorganizing and Rejuvenating CNNs for Increasing Model Capacity Utilization

In this paper, we propose self-reorganizing and rejuvenating convolution...
research
03/06/2023

Testing the Channels of Convolutional Neural Networks

Neural networks have complex structures, and thus it is hard to understa...
research
12/20/2016

Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale

In recent years, the research community has discovered that deep neural ...
research
02/17/2018

Towards Principled Design of Deep Convolutional Networks: Introducing SimpNet

Major winning Convolutional Neural Networks (CNNs), such as VGGNet, ResN...
research
05/26/2021

Probabilistic Selective Encryption of Convolutional Neural Networks for Hierarchical Services

Model protection is vital when deploying Convolutional Neural Networks (...

Please sign up or login with your details

Forgot password? Click here to reset