1. Introduction
Scaling artificial neural networks (ANNs) up to solve large complex problems achieved a big success in various machine learning problems. The backpropagation and stochastic gradient descent algorithms are conventional methods for training ANNs
(LeCun et al., 1998). An alternative approach, Neuroevolution (NE) (Floreano et al., 2008), employs evolutionary algorithms to optimize the topology and/or weights of the ANNs. The NE algorithms do not require the gradient information, and perform remarkably well in optimizing ANNs based on the direct interaction with their environment; specifically, in the cases where good decision instances are noisy or not known for supervised learning
(Floreano et al., 2008; Whitley et al., 1993).There are mainly two types of NE approaches: direct and indirect encoding (Floreano et al., 2008). Direct encoding aims to evolve the network parameters directly representing them within the genotype of the individuals; whereas, indirect encoding aims to evolve the specifications to define the developmental process of an ANN represented within the genotype. The indirect encoding methods can help improving the scalability of the evolutionary process for large networks, since they can reduce the parameter size. On the other hand, the NE with direct encoding presents a challenging opportunity for stimulating the research in largescale optimization, but also contributes to understanding the evolutionary dynamics of ANNs by suggesting successful evolutionary strategies to evolve ANNs.
The task of evolving directencoded large networks is challenging due to 1) the scalability of the evolutionary methods to perform the optimization process efficiently on highdimensional search spaces, and 2) the time requirement for evaluating the individuals on a large number of training instances. The Cooperative Coevolution (CC) is an effective approach for optimizing largescale problems (Potter and De Jong, 1994); and the Limited Evaluation (LE) is an advantageous method for reducing the number of instances of fitness evaluations (Morse and Stanley, 2016). In this work, we propose a Limited Evaluation Cooperative Coevolutionary Differential Evolution (LECCDE) algorithm that employs the CC and LE approaches to perform accelerated evolution in optimizing highdimensional ANNs with direct encoding.
With respect to the previous works, the work presented in this paper contributes as follows: 1) it considers the postsynaptic neurons as the building blocks of an ANN, and performs the subcomponent decomposition of the CC scheme by assigning the presynaptic weights of each postsynaptic neuron to a subpopulation; 2) it demonstrates the effectiveness of the CC in optimizing largescale ANNs, and compares with the standard Differential Evolution (DE) optimization; 3) it shows that the LE scheme enhanced with the CC achieves better accuracy results than standard DE for evolving large networks, while reducing the time required for the fitness evaluation.
Three datasets were chosen to evaluate the performance of the proposed algorithm on supervised learning tasks. We used a fully connected feed forward ANNs with one hidden layer, with a total number of parameters in the order of thousands. We refer to these ANNs as “largescale” in the sense of NE with direct encoding, and to distinguish them from the specialized networks used in Deep Learning (DL) approaches
(LeCun et al., 2015).The rest of the paper is organized as follows: in Section 2, we provide the background knowledge and a brief literature review on the topics of DE, CC, and NE; in Section 3, we discuss the proposed algorithm in detail; in Section 4, we present the experimental setup; in Section 5, we provide the numerical results; and finally, in Section 6, we discuss the conclusions.
2. Related Work
In this section, we provide a brief overview of the background and related work.
2.1. Differential Evolution
The DE algorithm is a powerful yet simple populationbased search algorithm for continuous optimization (Storn and Price, 1997). A candidate solution set consists of individuals represented as
dimensional realvalued vectors
where integer . An initial population of individuals is randomly sampled from the domain ranges of each dimension and where is the th dimension of th individual.In each generation , an individual , called the target vector, is selected. The mutation and crossover operators are applied to generate a trial vector . The trial vector is evaluated, and replaced with the target vector by the selection operator, if the fitness value of the trial vector is greater than or equal to the target vector.
The mutation operator generates a mutant vector by perturbing a randomly selected vector using the scaled differences of the other two randomly selected vectors. The magnitude of the perturbation is controlled by the parameter called scale factor . This strategy is referred to as the “rand/1” strategy, and is provided by the following equation:
(1) 
where are mutually exclusive integers different from , and selected randomly from the range of . There are various alternative mutation strategies proposed in the literature (Das et al., 2016; Neri and Tirronen, 2010).
The crossover operator recombines the target vector with the mutant vector, controlled by the parameter called crossover rate . The binomial (uniform) and exponential crossover operators are the two most commonly used crossover operators (Price et al., 2006). The binomial crossover operator is provided in Equation (2):
(2) 
where integer refers to the th dimension of the vectors, functions and uniformly samples real and integer values within the specified ranges respectively.
The selection operator performs a comparison of the fitness values of the target and trial vectors, and replaces the target vector with the trial vector in the next generation if a better fitness value is achieved by the trial vector. This is referred to as synchronous update, since the replacements are performed at the end of the generation when the process for all individuals is complete. The asynchronous version of the update is implemented by performing the replacement immediately within the same generation. The asynchronous update allows a newly replaced trial vector to be used by other individuals within the same generation.
The settings of the parameters in DE plays an influential role in the behavior of the algorithm for balancing the tradeoff between the exploration and exploitation (Neri and Tirronen, 2010; Črepinšek et al., 2013). A recent survey by Karafotias et al. reviewed the approaches for parameter tuning and control in evolutionary algorithms (Karafotias et al., 2015). Neri and Tirronen surveyed the existing works in the literature and performed an empirical analysis of the strategies and parameters in DE; more recently, Das et al. reviewed the works in the literature on the selfadaptive parameter control in DE (Das et al., 2016).
2.2. Cooperative Coevolution
While the dimensionality of a problem increases, the performance of the evolutionary algorithms tend to decrease (Mahdavi et al., 2015; Liu et al., 2001). The CC schemes were proposed for scaling evolutionary algorithms to higher dimensions using a divideandconquer strategy. In the CC, the subcomponents of a largescale problem is decomposed and assigned to a number of subpopulations, that are evolved separately (Potter and De Jong, 1994). Cooperation in coevolution arises during the fitness evaluation, where the subcomponents are merged together to assign a global fitness score to a candidate solution.
The three aspects that play a key role in CC are problem decomposition, subcomponent evolution, and subcomponent coadaptation (Yang et al., 2008). The maximum number of subpopulations can be generated by splitting a dimensional problem into subgroups, assigning each subcomponent (dimension) to one subpopulation. Alternatively, the number of subcomponents in each subpopulation can be chosen arbitrarily to make the evolutionary optimization process manageable by reducing the dimensionality per subgroup. However, an arbitrary assignment of subcomponents may not be effective for solving nonseparable problems. Ideally, the problem should be decomposed in a way that the interdependency between the subcomponents in different subpopulations should be minimized.
The existing knowledge about the problem domain can be beneficial in the problem decomposition process. If the interdependencies of the subcomponents are known, the problem can be decomposed based on this knowledge. This also relates to the separability property of the problem. If the problem is separable, then the problem can be decomposed into its separable subcomponents. If there is no/uncertain knowledge of the problem domain, then automated methods can be used to identify the interactions of the subcomponents (Sun et al., 2017; Omidvar et al., 2014).
2.3. Neuroevolution
ANNs are computational models that are inspired by the central nervous system (De Castro, 2006). NE is a field that aims to optimize ANNs by using evolutionary computing methods (Floreano et al., 2008). The approaches suggested in NE can be grouped as direct and indirect encoding methods. One of the first examples of the direct encoding approaches evolved the connection weights of fixed topology networks by representing them within the genotype of the individual in the population (Whitley et al., 1990; Fogel et al., 1990).
Neuroevolution of Augmenting Topologies (NEAT) has been proposed to evolve both the topology and the weights of the networks starting from minimal networks and incrementally grow larger networks through the evolutionary process (Stanley and Miikkulainen, 2002). NEAT uses a global innovation counter to keep track of the history of changes, and to align the networks to generate more meaningful offspring as a result of the crossover operator.
Some of the works incorporate the CC scheme within Neuroevolution. The Symbiotic Adaptive Neuroevolution (SANE) evolves two separate populations, one for neurons and another for the network “blueprints”. The evolved network blueprints are used to determine which combinations of the neurons to use from the neuron population to generate a network (Moriarty, 1997). The Enforced SubPopulations (ESP) initiates a subpopulation for each neuron, and the genotype of these neurons encode the weights for incoming, outgoing and bias connections (Gomez, 2003)
; Cooperative Synapse Neuroevolution (CoSyNE) initiates a subpopulation for each connection
(Gomez et al., 2008).The indirect NE methods can help scaling evolutionary approaches for evolving large networks. Kitano (Kitano, 1990) suggested a grammatical graph encoding method, based on graph rewriting rules represented as individuals’ genotypes, to evolve the connectivity matrix of ANNs. Koutnik et al.,
proposed using lossy compression techniques to reduce the highdimensional parameters of the networks by transforming their parameters to the frequency domain using transformation functions such as the Fourier Transform and the Discrete Cosine Transform. In this case the evolutionary process is performed on a few significant coefficients on the frequency domain
(Koutnik et al., 2010). Gruau suggested a developmental method that evolves treestructured programs to specify the instructions to grow ANNs based on cell division and differentiation(Gruau, 1994). Stanley et al., proposed a HypercubeBased Encoding method that uses Compositional Pattern Producing Networks (CPPNs) to assign the connection weights between neurons as a function of their locations (Stanley et al., 2009; Stanley, 2007).The ANN architectures used in DL are often engineered for certain tasks in computer vision and signal processing
(LeCun et al., 2015). In this case the connection weights are typically trained using the backpropagation. On the other hand, there are hyperparameters for specifying the architecture and learning algorithms that play a role in the performance of network; thus, deep NE approaches have been suggested for optimizing the hyperparameters of the deep neural networks efficiently (Miikkulainen et al., 2017; Real et al., 2017). Some recent work focuses on scalable evolutionary approaches for optimizing the connection weights of the networks. Salimans et al.,used Evolution Strategies (ES) to optimize the connection weights of the Convolutional Neural Networks (CNNs) for reinforcement learning in MuJoCo and Atari environments
(Salimans et al., 2017). The CNNs are a specific type of large ANN topologies that are specifically designed for image processing/recognition tasks in DL. Zhang et al., compared the ES proposed by Salimans et al. with the stochastic gradient descent for training CNNs on a large handwritten digit dataset, MNIST, and showed that the ES can achieve the stateoftheart accuracy results (Zhang et al., 2017).Another scalability challenge for the NE is the fitness evaluation that can be computationally expensive, especially when there are large numbers of training instances to evaluate. Morse and Stanley proposed an approach called Limited Evaluation Evolutionary Algorithm (LEEA), inspired by the batch training in the stochastic gradient descent algorithm. The LEEA performs fitness evaluations over a small number of training instances (batches), and uses accumulated fitness values that are inherited from the parent generation to the offspring generation between batches (Morse and Stanley, 2016). We adopt the LEEA approach in our algorithm, and discuss the approach in more detail in Section 3.
3. The Proposed Algorithm
The implementation details of the LECCDE algorithm are given in Algorithm 1. The algorithm is composed of the CC and LE schemes to decompose a largescale continuous optimization task, and speed up the fitness evaluation process.
The CC scheme in LECCDE uses a heuristic to decompose the parameters of a highdimensional ANN, i.e. the postsynaptic neurons are assumed to be the building blocks of the ANN, and are decomposed into subpopulations and evolved separately. Thus, the algorithm initiates
subpopulations for each postsynaptic neuron, where each subpopulation consists of individuals. Each individual represents the presynaptic connection weights (see Appx. A).From the subpopulations that contain of individuals, there are ANNs that can be constructed. To find the average fitness of each individual, all possible network combinations need to be evaluated. Since this number is quite large, we randomly sample times an individual from each subpopulation, construct a global network, evaluate it, and add the fitness value of the network to the fitness values of each individual that was part of the network (Gomez et al., 2008). At the end of this procedure, the fitness value of each individual is normalized to find the average fitness value, dividing by the number of time each individual is selected. The fitness of the individuals that were not selected during the sampling process set to 0. The individual with the maximum fitness from each subpopulation is then selected to construct the global ANN solution . Finally, the performance of the global solution on the validation instances is found by evaluating on the validation set.
The main loop of the algorithm iterates over all the batches. A batch is a small subset of the training instances used in the LE scheme (Morse and Stanley, 2016). In particular, batches are generated by randomly assigning each training instance to a batch.
The fitness score of the target vector on the current batch is found by replacing it within its corresponding part in the global solution, and evaluating the global solution on the current batch. Subsequently, the fitness of the target vector is adjusted using the asexual reproduction rule (see Appx. A.2).
The fitness of the trial vector is computed in a similar fashion, by first replacing its corresponding part within the global solution, and then evaluating the global solution on the current batch. Since the mutant vector is composed of three randomly selected individuals , the fitness value of the mutant vector is computed by taking their average. The fitness value of the trial vector is found using the sexual reproduction rule (see Appx. A.2).
The selection operator copies the trial vector and its fitness to a temporary set if its fitness value is greater than or equal to the fitness value of the target vector; otherwise, the target value and its fitness are copied. After all the computations are completed for all individuals in the subpopulation, the subpopulation is updated simultaneously by copying back the individuals and their fitnesses from the temporary sets.
After each subpopulation update, the individual with the highest fitness value in the subpopulation is copied back to the corresponding part of the global solution . The global solution is evaluated on the validation set, and the one that performed the best is stored and provided as a the final result of the algorithm.
4. Experimental setup
Our experimental setup is designed to focus on the following questions:

Do the ANNs that are evolved using the Cooperative Coevolutionary DE algorithm with our subpopulation assignment heuristic achieve a better classification accuracy than the ANNs that are evolved by the standard DE algorithm?

Does the LE scheme applied to DE reduce the runtime of the algorithm, without decreasing the classification accuracy of the evolved ANNs?
To answer these questions, we compare the results of the ANNs optimized by four algorithms, DE, LEDE, CCDE, and LECCDE, on three datasets with various sizes. The details for the implementation of the LECCDE are given in Algorithm 1. The CCDE and LEDE are implemented in a similar way, but, without the batch loop and the subpopulations, respectively. In standard DE, both batch training and subpopulations are not used. The LE algorithms require two evaluation per generation (target and trial vectors are evaluated on the current batch), while the algorithms without LE require one evaluation per generation. Regardless of this fact, the algorithms were run for the same number of function evaluations (FEs) for each dataset. For all experiments, we used “rand/1/bin” (“rand/1” mutation with binomial crossover) strategy with empirically fixed the parameter settings of and to and , respectively. We used individuals for the population size, except for one experiment that we performed on a larger population size consisting of individuals (see below). We set parameter to 5.
The three datasets used in the test process are listed in Table 1. These datasets were obtained from the Center for Machine Learning and Intelligent Systems dataset repository (Lichman, 2013). These datasets were chosen based on their number of features and instances, to show the relative performance of the algorithms in respect to the size of the dataset used. The Wisconsin breast cancer (WBC) dataset consists of 30 features, 2 classes, and 569 instances, the epileptic seizure recognition (ESR) consists of 178 features, 2 classes, and 4600 instances^{1}^{1}1The original epileptic seizure recognition dataset (Andrzejak et al., 2001) consists of 5 classes (first class for the measurements of the patients who had epileptic seizure and the remaining 4 classes for the measurements of the patients who did not have epileptic seizure), and 11500 instances (2300 for each class). To reduce the complexity of the task we took only the instances from the first and second classes with 2300 instances from each, thus considering 4600 instances in total., and the human activity recognition (HAR) dataset consists of 561 features, 6 classes, and 7144 instances (Anguita et al., 2013). The instances in each dataset were split into three groups (training, validation, and test) with ratios , , and
respectively. The fitness evaluations and selection process were performed on the train instances. The network that performs the best on the validation set is provided as the output of the algorithm, and evaluated on the test set. The fitness evaluation is based on the classification accuracy of the ANNs which is calculated by the number of correctly classified instances divided by the total number of instances.
For all datasets, we used fixedtopology fullyconnected feed forward ANNs with one hidden layer to perform the classification task (see Appx. A). The number of neurons within the hidden layer was kept constant at for all ANNs evolved for all datasets. Based on the architecture of the ANNs and the number of features in the datasets, the total number of parameters evolved are 1652, 9052, and 28406 for the WBC, ESR, and HAR respectively.
We used a batch size of 100 instances for the WBC, 500 for the ESR, and 500 for the HAR. The decay value (see Appx. A.2) is set to 0.2, as suggested by Morse and Stanley (Morse and Stanley, 2016). The maximum number of FEs was set to 50000 for the WBC, 300000 for the ESR, and 500000 for the HAR, based on the number of their parameters.
Datasets  Features  Classes  Instances  Parameters 

WBC  30  2  569  1652 
ESR  178  2  4600  9052 
HAR  561  6  7144  28406 
5. Numerical results
In this section, we present our experimental results. Each algorithm, with the specified settings, was run for 20 independent runs, and the median and the variance of train, validation and test accuracy were collected. All the accuracy results are shown with a precision of two digits.
Table 2 shows the results obtained from the WBC dataset. In this case we could not observe a significant difference on the results of the ANNs evolved by the four algorithms. On the test data, the CCDE appears to be performing better than others. On the other hand, we observe a difference on the runtime of the algorithm ( sec, in our computing environment^{2}^{2}2All algorithms were run, in singlecore, on an Intel Xeon E5 3.5GHz computer.). The algorithms that employ LE and CC are less computationally expensive and run faster. For example, the runtime of DE is more than twice as big as that of LECCDE. This difference is less significant for the other algorithms, due to the size of the dataset. Even though all the algorithms are run for 50000 FEs for this dataset, the algorithms with LE performed evaluation on batches that are four times smaller than the whole set of training instances. However, since CCDE is run on the whole dataset, it appears that the CC improved its runtime possibly due to the computations of reducedsized vectors within each subpopulation.
Alg.  Train  Validation  Test  Runtime 

DE  94.74 2.2  97.65 0.8  95.29 2.2  
LEDE  95.99 1.2  98.82 0.6  95.29 2.3  
CCDE  96.49 1.3  97.65 0.8  96.47 1.8  
LECCDE  96.24 1.9  97.65 0.8  95.29 2.8 
Table 3 presents the results obtained from the ESR dataset. Based on the test data, CCDE appears to show better performance than the rest of the algorithms, while LECCDE follows it very closely. We observe the best running time with LECCDE ( sec).
Alg.  Train  Validation  Test  Runtime 

DE  90.50 1.3  89.86 1.2  89.57 1.7  
LEDE  92.86 0.9  92.25 0.8  91.30 0.9  
CCDE  93.94 0.8  93.33 0.3  92.17 0.9  
LECCDE  93.98 0.6  92.61 0.5  91.88 1.0 
Table 4 shows the results obtained from the HAR dataset. On this dataset, we observe a significant accuracy improvement when the CC scheme is used. The performance of CCDE and LECCDE are approximately %1520 better than the algorithms that do not use the CC. Also, CCDE appears to be slightly better than LECCDE. On the other hand, we observe a significant runtime improvement when the LE scheme is used. The algorithms with the LE scheme run approximately four times faster than the algorithms that do not use the LE ( sec). Also, LECCDE appears to produce the smallest variance on th train accuracy.
Alg.  Train  Validation  Test  Runtime 

DE  70.06 2.9  70.06 2.7  68.38 3.0  
LEDE  77.5 5.2  77.99 4.8  76.96 4.8  
CCDE  94.01 0.8  92.72 0.7  92.4 1.0  
LECCDE  93.58 0.6  93.19 0.5  92.16 0.7 
Finally, in Table 5, we report an additional experiment on the population size. In this case, we used a population size of 100 on the ESR dataset. When the population size increases (comparing to the Table 3), the accuracy results decrease. This may be due to the number of FEs needed for the convergence of the algorithm: in other words, when the population increases, the number of FEs needed for the convergence may increase. Moreover, we observe that CCDE and LECCDE perform significantly better than DE and LEDE. This may suggest that the CC increased the convergence speed. With respect to the running time of the algorithms, we observe the similar pattern observed in Table 3 ( sec).
Overall, CCDE appears to perform better than LECCDE due to the fact that it has the complete information for evaluating the individuals since it uses the entire set of training instances. However, CCDE comes with a larger runtime tradeoff than LECCDE, which can make the difference with large datasets (e.g. for the HAR dataset the LECCDE runs on average four times faster). Also, increasing the number of evaluations or batch size can improve the performance of the LECCDE. For comparison, we performed two additional experiments with LECCDE, with the same settings used to produce the results in the Table 4, except the number of FEs and batch size. In the first experiment, we used 900000 FEs and observed that the ANNs the LECCDE optimize perform on training, validation and test sets on average 95.78, 94.31, and 93.28 respectively. This is almost %1 higher than the the performance observed in 4. On the other hand, the runtime of the algorithm is now , which is still times faster than the runtime of CCDE. In the second experiment, we used a batch size of 1000, and we observed that the algorithm performs on average 96.60, 93.84, and 93.38 on training, validation, and test datasets, with a runtime of . These two additional experiments show an interesting tradeoff between the batch size and the number of evaluations. Although the two additional experiments have similar runtime, the second experiment appears to produce better results.
Alg.  Train  Validation  Test  Runtime 

DE  81.99 1.4  82.61 1.2  80.65 1.9  
LEDE  80.25 1.4  81.59 0.8  80.14 2.2  
CCDE  91.65 1.3  91.45 0.7  90.29 0.9  
LECCDE  91.27 0.8  90.58 0.7  88.99 1.1 
Figure 1 shows the overall comparison of the runtime of LEDE, CCDE, and DE relative to LECCDE on the three datasets. The axis shows the dataset, and the axis shows the increase in the runtime of the algorithm. The LEDE is relatively stable across experimented datasets. On the other hand, the runtime of the CCDE and DE increases when the number of instances increases. This is because the algorithms with LE perform the same number of function evaluations, on a smaller number of instances, which produces a clear advantage in terms of total runtime.
Figure 2 shows the accuracy trend of the ANNs on the training, validation, and test instances during one example run of the optimization process performed by LECCDE (only the range is shown on the axis, for the sake of clarity). The data collected from this specific run shows that the accuracy on the training data is almost always the highest. The accuracy results of the test data closely follows the validation accuracy, and it is even higher for some of generations.
Figure 3 shows the change of the validation accuracy during the evolutionary process of four algorithms on a single run (only the range is shown on the axis). Firstly, the lines that represent the results of the LEDE and LECCDE are shorter than those of the other algorithms because they consume the same number of FEs within a half number of generations, since they perform two FEs (trial and target vectors) per generation. We observe that LEDE improves the DE in terms of validation accuracy and convergence speed; however, it suffers from the lack of diversity within the population (for a population size of 20), which prevents it from finding better solutions after about 80000 FEs are consumed. On the other hand, CCDE appears not to suffer from the early convergence problem observed in the LEDE, while LECCDE appears to improve the speed of CCDE.
To summarize, our empirical analysis suggests positive answers to the questions posed in Section 4: (1) It appears more significantly on large dataset (in Table 4), or with a large population size (in Table 5), that the ANNs that are evolved using the CC scheme using our heuristic achieve a better classification accuracy than the ANNs that are evolved by the standard DE algorithm; and (2) all experiments on the three datasets (most significantly on the largest dataset in Table 4), show that the LE scheme applied to DE reduce the runtime of the algorithm considerably, without causing a degradation on the classification accuracy of the evolved ANNs.
To further assess the scalability of the proposed algorithm, we performed an additional experiment on the MNIST dataset (LeCun et al., 1998). We used the same ANN architecture that was used in the previous experiments. We provide the numerical results —which are not shown here for brevity— on the extended version of the paper available online^{3}^{3}3Supplementary results available at: https://arxiv.org/abs/1804.07234. For the same number of function evaluations, the computing time required for the LEDE and LECCDE is about 25 times less than the computing time required for the DE and CCDE. The LECCDE performs 8% better than LEDE. Overall, our preliminary results on MNIST show that the LECCDE achieves 90.80% classification accuracy on the test data, on average, which is about 4% lower than the backpropagation algorithm on the same ANN architecture. This may suggest that a better parameter tuning may be needed for the LECCDE to obtain results which are comparable to the stateoftheart.
6. Conclusions
In this work, we proposed the LECCDE algorithm that employs the LE and CC schemes to improve the accuracy and the runtime of the standard DE algorithm for largescale NE with direct encoding.
We performed experiments on four datasets, including a preliminary test on the MNIST dataset. Our results show that the CC scheme improves the performance of DE on the tested classification tasks. Moreover, we used the LE scheme to further improve the scalability of the method. Our results show that the LE scheme reduces the runtime of the algorithms, without affecting the performance. This reduction is due to the fact that the evaluation is performed on a small number of instances.
We used a heuristic in the CC scheme that decomposes the problem at the level of postsynaptic neurons. Thus, we evolve all the presynaptic weights of the postsynaptic neurons in different subpopulations. This decomposition approach aims to reduce the parameter size per subpopulation. For large datasets on the other hand, the number of parameters per subpopulation may still be large. Although this heuristic worked well, there may also be other decomposition heuristics that can be more effective. Alternatively, automatic methods can also be used for this purpose.
Another possibility for improving the results can be achieved by performing a sensitivity analysis. In this work, we did not experiment on the strategy and the parameters settings of the DE algorithm. Selfadaptive parameter control approaches can also be investigated to improve the performance of the results since these approaches can adjust the balance between the exploration and exploitation during the search process (Yaman et al., 2018; Das et al., 2016).
The methods proposed here can evolve only the ANNs with fixed topologies, it will be useful to extend these methods also to the network topology optimization.
Appendix A Neuroevolution
a.1. Direct Encoding and Network Computation
An example of a feed forward network (FFN) is shown in Figure (a)a where each node represents a neuron, and each edge represents a connection between two node, and the direction of each edge represents the direction of the information flow. A FFN consists of a number of input (), hidden (), and output () neurons ( and are bias neurons kept constant at 1) structured as input, hidden and output layers respectively (see Figure (a)a). Inspired by biological neural networks, the connections between the neurons are often called synapses. A neuron that is at the starting point of the directional edge is called a presynaptic neuron, and the neuron that is at the end point (arrow) of the directional edge is called a postsynaptic neuron.

Figure (b)b shows the genotype representation of the network given in Figure (a)a. Each synaptic weight is mapped directly to a gene in the genotype. The genotype is divided into its subcomponents consisting of the presynaptic weights of each postsynaptic neuron.
The activation of each neuron is updated using Equation (3) where is the activation of a postsynaptic neuron, is the activation of the th presynaptic neuron and is the connection between them, is the bias of the postsynaptic neuron, and
is an activation function given in (
4) (De Castro, 2006).(3) 
(4) 
a.2. Limited Evaluation
When the evaluation is performed episodically on a small subset of the whole training instances (batches), it is required to keep track of the individuals that performed well on the previous episodes. The LE scheme aims to adjust the fitness of the offspring by taking into account the success of its parents by fitness inheritance. The sexual and asexual reproduction rules are provided in Equations (5) and (6) respectively (Morse and Stanley, 2016).
(5) 
(6) 
where, is the adjusted fitness of the offspring, is the parent of its parent, is the actual fitness of the offspring on current batch of the training instances, and is a constant value for adjusting the weight of the previous fitness evaluations. The sexual reproduction method consists of two parents.
Acknowledgements.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No: 665347. 
References
 (1)
 Andrzejak et al. (2001) Ralph G Andrzejak, Klaus Lehnertz, Florian Mormann, Christoph Rieke, Peter David, and Christian E Elger. 2001. Indications of nonlinear deterministic and finitedimensional structures in time series of brain electrical activity: Dependence on recording region and brain state. Physical Review E 64, 6 (2001), 061907.
 Anguita et al. (2013) Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis ReyesOrtiz. 2013. A Public Domain Dataset for Human Activity Recognition using Smartphones.. In ESANN.
 Črepinšek et al. (2013) Matej Črepinšek, ShihHsi Liu, and Marjan Mernik. 2013. Exploration and exploitation in evolutionary algorithms: A survey. ACM Computing Surveys (CSUR) 45, 3 (2013), 35.
 Das et al. (2016) Swagatam Das, Sankha Subhra Mullick, and Ponnuthurai N Suganthan. 2016. Recent advances in differential evolution–an updated survey. Swarm and Evolutionary Computation 27 (2016), 1–30.
 De Castro (2006) Leandro Nunes De Castro. 2006. Fundamentals of natural computing: basic concepts, algorithms, and applications. CRC Press.
 Floreano et al. (2008) Dario Floreano, Peter Dürr, and Claudio Mattiussi. 2008. Neuroevolution: from architectures to learning. Evolutionary Intelligence 1, 1 (2008), 47–62.
 Fogel et al. (1990) David B Fogel, Lawrence J Fogel, and VW Porto. 1990. Evolving neural networks. Biological cybernetics 63, 6 (1990), 487–493.
 Gomez et al. (2008) Faustino Gomez, Jürgen Schmidhuber, and Risto Miikkulainen. 2008. Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research 9, May (2008), 937–965.
 Gomez (2003) Faustino John Gomez. 2003. Robust nonlinear control through neuroevolution. Ph.D. Dissertation. University of Texas at Austin USA.
 Gruau (1994) Frédéric Gruau. 1994. Automatic definition of modular neural networks. Adaptive behavior 3, 2 (1994), 151–183.
 Karafotias et al. (2015) Giorgos Karafotias, Mark Hoogendoorn, and Ágoston E Eiben. 2015. Parameter control in evolutionary algorithms: Trends and challenges. IEEE Transactions on Evolutionary Computation 19, 2 (2015), 167–187.

Kitano (1990)
Hiroaki Kitano.
1990.
Designing neural networks using genetic algorithms with graph generation system.
Complex systems 4, 4 (1990), 461–476.  Koutnik et al. (2010) Jan Koutnik, Faustino Gomez, and Jürgen Schmidhuber. 2010. Evolving neural networks in compressed weight space. In Proceedings of the 12th annual conference on Genetic and evolutionary computation. ACM, 619–626.
 LeCun et al. (2015) Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436.
 LeCun et al. (1998) Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradientbased learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.
 Lichman (2013) M. Lichman. 2013. UCI Machine Learning Repository. (2013). http://archive.ics.uci.edu/ml
 Liu et al. (2001) Yong Liu, Xin Yao, Qiangfu Zhao, and Tetsuya Higuchi. 2001. Scaling up fast evolutionary programming with cooperative coevolution. In Evolutionary Computation, 2001. Proceedings of the 2001 Congress on, Vol. 2. Ieee, 1101–1108.
 Mahdavi et al. (2015) Sedigheh Mahdavi, Mohammad Ebrahim Shiri, and Shahryar Rahnamayan. 2015. Metaheuristics in largescale global continues optimization: A survey. Information Sciences 295 (2015), 407–428.
 Miikkulainen et al. (2017) Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Dan Fink, Olivier Francon, Bala Raju, Arshak Navruzyan, Nigel Duffy, and Babak Hodjat. 2017. Evolving deep neural networks. arXiv preprint arXiv:1703.00548 (2017).
 Moriarty (1997) David Eric Moriarty. 1997. Symbiotic evolution of neural networks in sequential decision tasks. Ph.D. Dissertation. University of Texas at Austin USA.
 Morse and Stanley (2016) Gregory Morse and Kenneth O Stanley. 2016. Simple evolutionary optimization can rival stochastic gradient descent in neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference 2016. ACM, 477–484.
 Neri and Tirronen (2010) Ferrante Neri and Ville Tirronen. 2010. Recent advances in differential evolution: a survey and experimental analysis. Artificial Intelligence Review 33, 12 (2010), 61–106.
 Omidvar et al. (2014) Mohammad Nabi Omidvar, Xiaodong Li, Yi Mei, and Xin Yao. 2014. Cooperative coevolution with differential grouping for large scale optimization. IEEE Transactions on evolutionary computation 18, 3 (2014), 378–393.
 Potter and De Jong (1994) Mitchell A Potter and Kenneth A De Jong. 1994. A cooperative coevolutionary approach to function optimization. In International Conference on Parallel Problem Solving from Nature. Springer, 249–257.
 Price et al. (2006) Kenneth Price, Rainer M Storn, and Jouni A Lampinen. 2006. Differential evolution: a practical approach to global optimization. Springer Science & Business Media.
 Real et al. (2017) Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Quoc Le, and Alex Kurakin. 2017. Largescale evolution of image classifiers. arXiv preprint arXiv:1703.01041 (2017).
 Salimans et al. (2017) Tim Salimans, Jonathan Ho, Xi Chen, and Ilya Sutskever. 2017. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864 (2017).
 Shi et al. (2005) Yanjun Shi, Hongfei Teng, and Ziqiang Li. 2005. Cooperative coevolutionary differential evolution for function optimization. In International Conference on Natural Computation. Springer, 1080–1088.
 Stanley (2007) Kenneth O Stanley. 2007. Compositional pattern producing networks: A novel abstraction of development. Genetic programming and evolvable machines 8, 2 (2007), 131–162.
 Stanley et al. (2009) Kenneth O Stanley, David B D’Ambrosio, and Jason Gauci. 2009. A hypercubebased encoding for evolving largescale neural networks. Artificial life 15, 2 (2009), 185–212.
 Stanley and Miikkulainen (2002) Kenneth O Stanley and Risto Miikkulainen. 2002. Evolving neural networks through augmenting topologies. Evolutionary computation 10, 2 (2002), 99–127.
 Storn and Price (1997) Rainer Storn and Kenneth Price. 1997. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of global optimization 11, 4 (1997), 341–359.
 Sun et al. (2017) Yuan Sun, Michael Kirley, and Saman K Halgamuge. 2017. A Recursive Decomposition Method for Large Scale Continuous Optimization. IEEE Transactions on Evolutionary Computation (2017).
 Whitley et al. (1993) Darrell Whitley, Stephen Dominic, Rajarshi Das, and Charles W Anderson. 1993. Genetic reinforcement learning for neurocontrol problems. Machine Learning 13, 23 (1993), 259–284.
 Whitley et al. (1990) Darrell Whitley, Timothy Starkweather, and Christopher Bogart. 1990. Genetic algorithms and neural networks: Optimizing connections and connectivity. Parallel computing 14, 3 (1990), 347–361.
 Yaman et al. (2018) Anil Yaman, Giovanni Iacca, Matt Coler, George Fletcher, and Mykola Pechenizkiy. 2018. Multistrategy differential evolution. In International Conference on the Applications of Evolutionary Computation. Springer, 617–633.
 Yang et al. (2008) Zhenyu Yang, Ke Tang, and Xin Yao. 2008. Large scale evolutionary optimization using cooperative coevolution. Information Sciences 178, 15 (2008), 2985–2999.
 Zhang et al. (2017) Xingwen Zhang, Jeff Clune, and Kenneth O Stanley. 2017. On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent. arXiv preprint arXiv:1712.06564 (2017).
Appendix B Extended experiments and results
This section presents our preliminary results of the experiments performed on the MNIST dataset using the DE, LEDE, CCDE and LECCDE. The MNIST dataset consists of 60000 samples of 28 by 28 grayscale image instances of handwritten numbers between 09. Thus, the size of the input and output are 784 and 10 when each image pixel and its class label are considered as an input and output respectively.
We used the same architecture of the artificial neural networks that were used for the experiments performed on the other datasets (feed forward artificial neural networks with one hidden layer consisting of 50 neurons). Thus, the total number of parameters of the networks optimized for the MNIST is 47710. The parameters of the Differential Evolution algorithm are also initialized using the same settings used for the other experiments except for batch size, number of individuals in each subpopulation and the maximum number of function evaluations. Since MNIST is larger than the tested other datasets, we used a a batch size of 1000, a population size of 60 and a maximum number of evaluations set to .
Table 6 shows the training, validation and test accuracy results of the ANNs trained for the MNIST dataset. Each variant of the algorithm was executed for the same number of function evaluations. The total time required for computing every other algorithm is shown in relation to the computing time required for the LECCDE where
seconds that is approximately 19 hours on a singlecore Intel Xeon E5 3.5GHz computer. Due to time constraints, we were able to perform 3 independent runs for the LEDE and LECCDE, and a single partial run for the DE and CCDE. Thus, on DE and CCDE we report their accuracy at 12% of their total allocated computing time (the total computing times of DE and CCDE are estimated based on their current execution progress).
Alg.  Train  Validation  Test  Runtime 

DE (% 12)  61.60  61.26  62.52  
LEDE  82.68 0.36  82.01 0.75  82.23  
CCDE (% 12)  62.40  61.80  63.20  
LECCDE  91.79 0.28  91.01 0.63  90.80 0.15 
We observe a significant advantage in using the LE scheme on MNIST from the computing time point of view: indeed, the DE and CCDE implementations of the algorithm require a computing time that is 25 times bigger than the computing time required by the corresponding algorithms that make use of the LE scheme.
Figure 5 illustrates the change of the validation accuracy of the evolved ANNs using the LECCDE during an evolutionary process. The speed of the accuracy improvements slows down around 88%  90% level. The best validation accuracy achieved during this evolutionary run was 91.62%.
Comments
There are no comments yet.