The Heterogeneity Hypothesis: Finding Layer-Wise Dissimilated Network Architecture

06/29/2020
by   Yawei Li, et al.
8

In this paper, we tackle the problem of convolutional neural network design. Instead of focusing on the overall architecture design, we investigate a design space that is usually overlooked, adjusting the channel configurations of predefined networks. We find that this adjustment can be achieved by pruning widened baseline networks and leads to superior performance. Base on that, we articulate the “heterogeneity hypothesis”: with the same training protocol, there exists a layer-wise dissimilated network architecture (LW-DNA) that can outperform the original network with regular channel configurations under lower level of model complexity. The LW-DNA models are identified without added computational cost and training time compared with the original network. This constraint leads to controlled experiment which directs the focus to the importance of layer-wise specific channel configurations. Multiple sources of hints relate the benefits of LW-DNA models to overfitting, the relative relationship between model complexity and dataset size. Experiments are conducted on various networks and datasets for image classification, visual tracking and image restoration. The resultant LW-DNA models consistently outperform the compared baseline models.

READ FULL TEXT
research
07/24/2018

Convolutional Neural Networks In Classifying Cancer Through DNA Methylation

DNA Methylation has been the most extensively studied epigenetic mark. U...
research
06/13/2018

An image representation based convolutional network for DNA classification

The folding structure of the DNA molecule combined with helper molecules...
research
04/06/2020

Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio

Automatic designing computationally efficient neural networks has receiv...
research
04/30/2020

Out-of-the-box channel pruned networks

In the last decade convolutional neural networks have become gargantuan....
research
06/21/2023

Balanced Mixture of SuperNets for Learning the CNN Pooling Architecture

Downsampling layers, including pooling and strided convolutions, are cru...
research
11/12/2019

CALPA-NET: Channel-pruning-assisted Deep Residual Network for Steganalysis of Digital Images

Over the past few years, detection performance improvements of deep-lear...
research
04/26/2023

Sparsified Model Zoo Twins: Investigating Populations of Sparsified Neural Network Models

With growing size of Neural Networks (NNs), model sparsification to redu...

Please sign up or login with your details

Forgot password? Click here to reset