Convolution Neural Network Hyperparameter Optimization Using Simplified Swarm Optimization

by   Wei-Chang Yeh, et al.

Among the machine learning approaches applied in computer vision, Convolutional Neural Network (CNN) is widely used in the field of image recognition. However, although existing CNN models have been proven to be efficient, it is not easy to find a network architecture with better performance. Some studies choose to optimize the network architecture, while others chose to optimize the hyperparameters, such as the number and size of convolutional kernels, convolutional strides, pooling size, etc. Most of them are designed manually, which requires relevant expertise and takes a lot of time. Therefore, this study proposes the idea of applying Simplified Swarm Optimization (SSO) on the hyperparameter optimization of LeNet models while using MNIST, Fashion MNIST, and Cifar10 as validation. The experimental results show that the proposed algorithm has higher accuracy than the original LeNet model, and it only takes a very short time to find a better hyperparameter configuration after training. In addition, we also analyze the output shape of the feature map after each layer, and surprisingly, the results were mostly rectangular. The contribution of the study is to provide users with a simpler way to get better results with the existing model., and this study can also be applied to other CNN architectures.



There are no comments yet.


page 13

page 16

page 17


Optimization of Convolutional Neural Network using Microcanonical Annealing Algorithm

Convolutional neural network (CNN) is one of the most prominent architec...

Optimization of Convolutional Neural Network Using the Linearly Decreasing Weight Particle Swarm Optimization

Convolutional neural network (CNN) is one of the most frequently used de...

ExperienceThinking: Hyperparameter Optimization with Budget Constraints

The problem of hyperparameter optimization exists widely in the real lif...

Optimization of Residual Convolutional Neural Network for Electrocardiogram Classification

The interpretation of the electrocardiogram (ECG) gives clinical informa...

Deep Genetic Network

Optimizing a neural network's performance is a tedious and time taking p...

Evaluating Generic Auto-ML Tools for Computational Pathology

Image analysis tasks in computational pathology are commonly solved usin...

Convolutional Neural Network Simplification with Progressive Retraining

Kernel pruning methods have been proposed to speed up, simplify, and imp...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.