Multi-objective Evolutionary Approach for Efficient Kernel Size and Shape for CNN

06/28/2021
by   Ziwei Wang, et al.
0

While state-of-the-art development in CNN topology, such as VGGNet and ResNet, have become increasingly accurate, these networks are computationally expensive involving billions of arithmetic operations and parameters. To improve the classification accuracy, state-of-the-art CNNs usually involve large and complex convolutional layers. However, for certain applications, e.g. Internet of Things (IoT), where such CNNs are to be implemented on resource-constrained platforms, the CNN architectures have to be small and efficient. To deal with this problem, reducing the resource consumption in convolutional layers has become one of the most significant solutions. In this work, a multi-objective optimisation approach is proposed to trade-off between the amount of computation and network accuracy by using Multi-Objective Evolutionary Algorithms (MOEAs). The number of convolution kernels and the size of these kernels are proportional to computational resource consumption of CNNs. Therefore, this paper considers optimising the computational resource consumption by reducing the size and number of kernels in convolutional layers. Additionally, the use of unconventional kernel shapes has been investigated and results show these clearly outperform the commonly used square convolution kernels. The main contributions of this paper are therefore a methodology to significantly reduce computational cost of CNNs, based on unconventional kernel shapes, and provide different trade-offs for specific use cases. The experimental results further demonstrate that the proposed method achieves large improvements in resource consumption with no significant reduction in network performance. Compared with the benchmark CNN, the best trade-off architecture shows a reduction in multiplications of up to 6X and with slight increase in classification accuracy on CIFAR-10 dataset.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 18

page 19

page 20

page 21

page 22

page 23

10/02/2019

A Pre-defined Sparse Kernel Based Convolution for Deep CNNs

The high demand for computational and storage resources severely impede ...
01/28/2021

Evolutionary Neural Architecture Search Supporting Approximate Multipliers

There is a growing interest in automated neural architecture search (NAS...
10/02/2019

A Pre-defined Sparse Kernel Based Convolutionfor Deep CNNs

The high demand for computational and storage resources severely impede ...
07/10/2019

Dual Dynamic Inference: Enabling More Efficient, Adaptive and Controllable Deep Inference

State-of-the-art convolutional neural networks (CNNs) yield record-break...
11/18/2019

Grassmannian Packings in Neural Networks: Learning with Maximal Subspace Packings for Diversity and Anti-Sparsity

Kernel sparsity ("dying ReLUs") and lack of diversity are commonly obser...
03/21/2019

Evolving Deep Neural Networks by Multi-objective Particle Swarm Optimization for Image Classification

In recent years, convolutional neural networks (CNNs) have become deeper...
08/11/2019

ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks

As designing appropriate Convolutional Neural Network (CNN) architecture...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.