Multigrid-in-Channels Neural Network Architectures

11/17/2020
by   Moshe Eliasof, et al.
0

We present a multigrid-in-channels (MGIC) approach that tackles the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs). It has been shown that there is a redundancy in standard CNNs, as networks with light or sparse convolution operators yield similar performance to full networks. However, the number of parameters in the former networks also scales quadratically in width, while in the latter case, the parameters typically have random sparsity patterns, hampering hardware efficiency. Our approach for building CNN architectures scales linearly with respect to the network's width while retaining full coupling of the channels as in standard CNNs. To this end, we replace each convolution block with its MGIC block utilizing a hierarchy of lightweight convolutions. Our extensive experiments on image classification, segmentation, and point cloud classification show that applying this strategy to different architectures like ResNet and MobileNetV3 considerably reduces the number of parameters while obtaining similar or better accuracy. For example, we obtain 76.1 parameters and FLOPs to MobileNetV3.

READ FULL TEXT

page 2

page 6

research
06/11/2020

Multigrid-in-Channels Architectures for Wide Convolutional Neural Networks

We present a multigrid approach that combats the quadratic growth of the...
research
06/08/2016

Convolutional Neural Fabrics

Despite the success of CNNs, selecting the optimal architecture for a gi...
research
09/30/2018

Benchmarks of ResNet Architecture for Atrial Fibrillation Classification

In this work we apply variations of ResNet architecture to the task of a...
research
02/15/2022

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks

CNN architectures are generally heavy on memory and computational requir...
research
09/07/2019

Exploring Temporal Differences in 3D Convolutional Neural Networks

Traditional 3D convolutions are computationally expensive, memory intens...
research
09/25/2020

Tied Block Convolution: Leaner and Better CNNs with Shared Thinner Filters

Convolution is the main building block of convolutional neural networks ...
research
04/17/2020

Finding the Optimal Network Depth in Classification Tasks

We develop a fast end-to-end method for training lightweight neural netw...

Please sign up or login with your details

Forgot password? Click here to reset