DeepAI AI Chat
Log In Sign Up

A Pre-defined Sparse Kernel Based Convolution for Deep CNNs

by   Souvik Kundu, et al.

The high demand for computational and storage resources severely impede the deployment of deep convolutional neural networks (CNNs) in limited-resource devices. Recent CNN architectures have proposed reduced complexity versions (e.g. SuffleNet and MobileNet) but at the cost of modest decreases inaccuracy. This paper proposes pSConv, a pre-defined sparse 2D kernel-based convolution, which promises significant improvements in the trade-off between complexity and accuracy for both CNN training and inference. To explore the potential of this approach, we have experimented with two widely accepted datasets, CIFAR-10 and Tiny ImageNet, in sparse variants of both the ResNet18 and VGG16 architectures. Our approach shows a parameter count reduction of up to 4.24x with modest degradation in classification accuracy relative to that of standard CNNs. Our approach outperforms a popular variant of ShuffleNet using a variant of ResNet18 with pSConv having 3x3 kernels with only four of nine elements not fixed at zero. In particular, the parameter count is reduced by 1.7x for CIFAR-10 and 2.29x for Tiny ImageNet with an increased accuracy of  4


A Pre-defined Sparse Kernel Based Convolutionfor Deep CNNs

The high demand for computational and storage resources severely impede ...

Multi-objective Evolutionary Approach for Efficient Kernel Size and Shape for CNN

While state-of-the-art development in CNN topology, such as VGGNet and R...

Reducing Deep Network Complexity with Fourier Transform Methods

We propose a novel way that uses shallow densely connected neuron networ...

Pre-defined Sparsity for Low-Complexity Convolutional Neural Networks

The high energy cost of processing deep convolutional neural networks im...

Penetrating the Fog: the Path to Efficient CNN Models

With the increasing demand to deploy convolutional neural networks (CNNs...

Memory Bounded Deep Convolutional Networks

In this work, we investigate the use of sparsity-inducing regularizers d...

Efficient Winograd or Cook-Toom Convolution Kernel Implementation on Widely Used Mobile CPUs

The Winograd or Cook-Toom class of algorithms help to reduce the overall...