Large Scale Evolution of Convolutional Neural Networks Using Volunteer Computing

03/15/2017
by   Travis Desell, et al.
0

This work presents a new algorithm called evolutionary exploration of augmenting convolutional topologies (EXACT), which is capable of evolving the structure of convolutional neural networks (CNNs). EXACT is in part modeled after the neuroevolution of augmenting topologies (NEAT) algorithm, with notable exceptions to allow it to scale to large scale distributed computing environments and evolve networks with convolutional filters. In addition to multithreaded and MPI versions, EXACT has been implemented as part of a BOINC volunteer computing project, allowing large scale evolution. During a period of two months, over 4,500 volunteered computers on the Citizen Science Grid trained over 120,000 CNNs and evolved networks reaching 98.32 accuracy on the MNIST handwritten digits dataset. These results are even stronger as the backpropagation strategy used to train the CNNs was fairly rudimentary (ReLU units, L2 regularization and Nesterov momentum) and these were initial test runs done without refinement of the backpropagation hyperparameters. Further, the EXACT evolutionary strategy is independent of the method used to train the CNNs, so they could be further improved by advanced techniques like elastic distortions, pretraining and dropout. The evolved networks are also quite interesting, showing "organic" structures and significant differences from standard human designed architectures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/17/2018

Accelerating the Evolution of Convolutional Neural Networks with Node-Level Mutations and Epigenetic Weight Initialization

This paper examines three generic strategies for improving the performan...
research
08/18/2021

Combining Neuro-Evolution of Augmenting Topologies with Convolutional Neural Networks

Current deep convolutional networks are fixed in their topology. We exp...
research
09/10/2018

Finding Better Topologies for Deep Convolutional Neural Networks by Evolution

Due to the nonlinearity of artificial neural networks, designing topolog...
research
02/06/2019

Investigating RNN Memory using Neuro-Evolution: Investigating Recurrent Neural Network Memory Structures using Neuro-Evolution

This paper presents a new algorithm, Evolutionary eXploration of Augment...
research
03/27/2018

Neuroevolution for RTS Micro

This paper uses neuroevolution of augmenting topologies to evolve contro...
research
04/20/2020

Hot-Starting the Ac Power Flow with Convolutional Neural Networks

Obtaining good initial conditions to solve the Newton-Raphson (NR) based...
research
08/06/2018

On Optimizing Deep Convolutional Neural Networks by Evolutionary Computing

Optimization for deep networks is currently a very active area of resear...

Please sign up or login with your details

Forgot password? Click here to reset