Deep Genetic Network

Optimizing a neural network's performance is a tedious and time taking process, this iterative process does not have any defined solution which can work for all the problems. Optimization can be roughly categorized into - Architecture and Hyperparameter optimization. Many algorithms have been devised to address this problem. In this paper we introduce a neural network architecture (Deep Genetic Network) which will optimize its parameters during training based on its fitness. Deep Genetic Net uses genetic algorithms along with deep neural networks to address the hyperparameter optimization problem, this approach uses ideas like mating and mutation which are key to genetic algorithms which help the neural net architecture to learn to optimize its hyperparameters by itself rather than depending on a person to explicitly set the values. Using genetic algorithms for this problem proved to work exceptionally well when given enough time to train the network. The proposed architecture is found to work well in optimizing hyperparameters in affine, convolutional and recurrent layers proving to be a good choice for conventional supervised learning tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/19/2018

Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks

Most learning algorithms require the practitioner to manually set the va...
research
06/23/2020

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

Convolutional Neural Networks (CNN) have gained great success in many ar...
research
05/19/2018

GADAM: Genetic-Evolutionary ADAM for Deep Neural Network Optimization

Deep neural network learning can be formulated as a non-convex optimizat...
research
07/03/2019

HyperNOMAD: Hyperparameter optimization of deep neural networks using mesh adaptive direct search

The performance of deep neural networks is highly sensitive to the choic...
research
04/28/2023

Hyperparameter Optimization through Neural Network Partitioning

Well-tuned hyperparameters are crucial for obtaining good generalization...
research
12/30/2017

Towards co-evolution of fitness predictors and Deep Neural Networks

Deep neural networks proved to be a very useful and powerful tool with m...
research
03/14/2021

Use of static surrogates in hyperparameter optimization

Optimizing the hyperparameters and architecture of a neural network is a...

Please sign up or login with your details

Forgot password? Click here to reset