1 Motivation
Most genetic programming (GP) [1] systems don’t adapt or improve from solving one problem to the next. Any experience which could have been gained by the system is usually completely forgotten when that same system is applied to a subsequent problem, the system effectively starts from scratch each time.
For instance, consider the successful application of a classical GP system to a standard nbits Boolean function synthesis benchmark (such as the 6bits comparator as described in [3]). The population which produced the solution tree is not useful for solving any other nbits Boolean benchmark (such as the 6bits multiplexer). Therefore, in general, an entirely new and different population must be generated and undergo evolution for each different problem. This occurs because the system components which adapt to solve the problem (a population of trees in the case of classical GP) become so specialised that they are not useful for any other problem.
This paper addresses this issue by introducing the NodebyNode Growth Solver (NNGS) algorithm, which features a component called the controller, that can be improved from one problem to the next within a limited class of problems.
NNGS uses Semantic Backpropagation (SB)
and the controller, to grow a single Sexpression solution tree starting from the root node. Acting locally at each node, the controller makes explicit use of the target output data and input arguments data to determine the properties (i.e. operator type or argument, and semantics) of the subsequently generated child nodes.
The proofofconcept controller discussed in this paper constitutes a set of deterministic hand written rules and has been tested, as part of the NNGS algorithm, on several popular Boolean function synthesis benchmarks. This work aims to pave the way towards the use of a neural network as an adaptable controller and/or, separately, towards the use of metaGP for improving the controller component. In effect, the aim is to exploit the advantages of blackbox machine learning techniques to generate small and examinable program solutions.
The rest of this paper will proceed as follows: Section 2 outlines other related research. Section 3 details semantic backpropagation. A high level overview of the NNGS system is given in Section 4, and Section 5 describes the proofofconcept controller. Section 6 details the experiments conducted. The experimental results and a discussion is given in Section 7. Section 8 concludes with a description of potential future work.
2 Related Work
The isolation of useful subprograms/subfunctions is a related research theme in GP. However, in most studies subprograms are not reused across different problems. In [2] for instance, the hierarchical automatic function definition technique was introduced so as to facilitate the development of useful subfunctions whilst solving a problem. Machine learning was employed in [3] to analyse the internal behaviour (semantics) of GP programs so as to automatically isolate potentially useful problem specific subprograms.
SB was used in [4] to define intermediate subtasks for GP. Two GP search operator were introduced which semantically searched a library of subtrees which could be used to solve the subtasks. Similar work was carried out in [6, 7], however in these cases subtree libraries were smaller and static, and only a single tree was iteratively modified as opposed to a population of trees.
3 Semantic Backpropagation (SB)
Consider the output array produced by the root node of a solution tree, where each element within the array corresponds to the output from one fitness case. This output array is the semantics of the root node. If the solution is perfectly correct, the output array will correspond exactly to the target output array of the problem at hand. In a programmatic style, the output array of a general node will be denoted as and the output from fitness case will be denoted by .
Each node within a tree produces an output array, a feature which has been exploited in [3] to isolate useful subtrees. The simplest example of a tree (beyond a single node) is a triplet of nodes: a parent node , the left child node , and the right child node .
As a two fitness case example, suppose that a triplet is composed of a parent node representing the operator , a left child node representing input argument , and a right child node representing input argument . The output array of the parent node is given by:
(1)  
On the other hand, given the output array from the parent node of a triplet , it is possible to backpropagate the semantics so as to generate output arrays for the child nodes, if the reverse of the parent operator is defined carefully. This work will exclusively tackle function synthesis problems within the Boolean domain, and therefore, the standard [2, 3] set of Boolean operators will be used: , , , and .
Figure 1 gives function tables for the elementwise reverse operators: , , , and (their use with 1D arrays as input arguments follows as expected). As an example use of these operators the previous example will be worked in reverse: given the output array of , the arrays and are calculated as:
(2)  
or  
The hash symbol in this case indicates that either or will do. Note that two different possible values for and exist because . This feature occurs as a result of rows 4 and 5 of the function table. Note that each of the other reverse operators have similar features, specifically for: , , and .
Note also, that for any array loci in where , it is true that and . For example, .
Using the reverse operators in this way, output arrays can be assigned to the child nodes of any parent node. The child output arrays will depend on two decisions: Firstly, on the operator assigned to the parent node, as this is the operator that is reversed. And secondly, on the choices made (note the example above), at each loci, as to which of the two child output arrays contains the value. These decisions are made by the controller component.
Using these reverse operators for SB can only ever produce a pair of output arrays which are different from the problem target outputs in two ways. Firstly, the output arrays can be a flipped (using the gate on each bit) or an unflipped versions of the problem target outputs. Secondly, some elements of the output arrays will be elements.
4 NodebyNode Growth Solver (NNGS)
A visual representation of the NNGS algorithm can be seen in Fig. 2, which shows a snapshot of a partially generated solution tree. This tree, in it’s unfinished state, is composed of: and operators, an input argument labelled , and two unprocessed nodes. The basic role of the NNGS algorithm is to manage growing the solution tree by passing unprocessed nodes to the controller and substituting back the generated/returned node triplet.
Algorithm 1 gives a simple and more thorough explanation of NNGS. In line 2 the output values of the solution tree root node are set to the target output values of the problem at hand. The output values of a node are used, along with the reverse operators, by the controller (line 12) to generate the output values of the returned child nodes. The controller also sets the node type (they are either operators or input arguments) of the input parent node and generated child nodes.
Nodes which have been defined by the controller as input arguments (with labels: , , … etc.) can not have child nodes (they are by definition leaf nodes) and are therefore not processed further by the controller (line 8). When every branch of the tree ends in an input argument node, the algorithm halts.
Note that the controller may well generate a triplet where one or more of the child nodes require further processing. In this case the NNGS algorithm will pass these nodes back to the controller at a later stage before the algorithm ends. In effect, by using the controller component the NNGS algorithm simply writes out the solution tree.
5 ProofOfConcept Controller
Given an unprocessed node, the controller generates two child nodes and their output arrays using one of the four reverse operators. It also sets the operator type of the parent node to correspond with the chosen reverse operator that is used.
The ultimate goal of the controller is to assign an input argument to each generated child node. For example, suppose that the controller generates a child node with an output array and that an input argument is given by . In this case, can be assigned (can represent) the input argument because . The algorithm halts once each leaf node has been assigned an input argument.
Before giving a detailed description of the proofofconcept controller, there are a few important general points to stress: Firstly, the entire decision making process is deterministic. Secondly, the decision making process is greedy (the perceived best move is taken at each opportunity). Thirdly, the controller does not know the location of the input node within the solution tree. The controller has priori knowledge of the input argument arrays, the operators, and the reverse operators only. Furthermore, the controller, in its current state, does not memorise the results of it’s past decision making. In this regard, when processing a node, the controller has knowledge of that node’s output array only. In this way, the controller acts locally on each node. Multiple instances of the controller could act in parallel by processing all unprocessed nodes simultaneously.
5.1 Stepbystep
This subsection will give a stepbystep runthrough of the procedure undertaken by the proofofconcept controller. Figure 3 serves as a diagrammatic aid for each major step.
5.1.1 Step 1
Given an input (parent) node , and for each reverse operator in Table 1, the first step taken by the controller is to generate output arrays for each of the child nodes. In the example given in step 1 of Fig. 3 only the reverse operator is used. The reverse operator generates values in the child output arrays due to the following property . In this step, whenever the opportunity arises (regardless of the reverse operator employed), all generated values within the child output arrays will be placed in the output array of the right child node . For example in the case of : will be used and not .
Note that the reverse operators propagate all elements from parent to child nodes. This feature is exemplified in step 1 of Fig. 3 by the propagation of the value at locus 4 of to loci 4 of both and .
5.1.2 Step 2
By this step, the controller has generated four different (in general) arrays, one for each reverse operator. The goal for this step is to compare each of those arrays to each of the possible input argument arrays (, … etc). As an example, in step 2 of Fig. 3 the generated array is compared to the input argument array.
Two separate counts are made, one for the number of erroneous 0 argument values and one for the number of erroneous 1 argument values (highlighted in blue and red respectively in Fig. 3). Two further counts are made of the number of erroneous loci, for 0 and 1 input arguments values, which could have been values (and therefore not erroneous) had the controller not specified in step 1 that all values should be placed in the array whenever possible. These last two statistics will be denoted by and for 0 and 1 input arguments values respectively. These four statistics form an error table for each reverse operatorinput argument pair.
5.1.3 Step 3
In this step, the controller sorts the error tables by a number of statistics. Note that and are the number of remaining erroneous 0 argument values and erroneous 1 argument values respectively if all values were moved from the array to the array whenever possible. To simplify matters we note that
(3)  
otherwise  
Each error table is ranked by (in order, all increasing): , , , , and the number of values in . In a greedy fashion, the very best error table (lowest ranked) will be select for the next step (in Fig. 3 the error table is selected). Note that the ranked list of error tables might need to be revisited later from step 5.
5.1.4 Step 4
The error table selected in step 3 effectively serves as an instruction which details how the and
arrays should be modified. The goal of the controller is to move the minimum number of values from the array to the array such as to satisfy the error count for either ’s or ’s in one of the input arguments. In the example given in Fig. 3, two values in are swapped with 1’s in .
5.1.5 Step 5
In this step, the algorithm checks that the generated
and arrays do not exactly equal either the parent node or the grand parent node (if it exists). If this check fails, the algorithm reverts back to step 3 and chooses the next best error table.
5.1.6 Step 6
The final step of the algorithm is to appropriately set the operator type of given the final reverse operator that was used. In this step the algorithm also checks whether either (or both) of the child nodes can represent input arguments given their generated output arrays.
6 Experiments
The Boolean function synthesis benchmarks solved in this paper are standard benchmarks within GP research [2, 3, 4, 6]. They are namely: the comparator 6bits and 8bits (CmpXX), majority 6bits and 8bits (MajXX), multiplexer 6bits and 11bits (MuxXX), and evenparity 6bits, 8bit, 9bits, and 10bits (ParXX).
Their definitions are succinctly given in [3]:
“For an vbit comparator Cmp v, a program is required to return true if the v/2 least significant input bits encode a number that is smaller than the number represented by the v/2 most significant bits. In case of the majority Maj v problems, true should be returned if more that half of the input variables are true. For the multiplexer Mul v, the state of the addressed input should be returned (6bit multiplexer uses two inputs to address the remaining four inputs, 11bit multiplexer uses three inputs to address the remaining eight inputs). In the parity Par v problems, true should be returned only for an odd number of true inputs.”
The evenparity benchmark is often reported as the most difficult benchmark [2].
7 Results and Discussion
Run time [seconds]  Program size [nodes]  

NNGS  BP4A  ILTI  NNGS  BP4A  ILTI  RDO  
Cmp06  0.06  15  9  99  156  59  185 
Cmp08  0.86  220  20  465  242  140  538 
Maj06  0.19  36  10  271  280  71  123 
Maj08  3.09  2019*  27  1391  563*  236   
Mux06  0.21  10  9  333  117  47  215 
Mux11  226.98  9780  100  12373  303  153  3063 
Par06  0.35  233  17  515  356  435  1601 
Par08  5.73  3792*  622  2593  581*  1972   
Par09  25.11    5850  5709    4066   
Par10  120.56      12447       
The results are given in Table 1 and show that the NNGS algorithm finds solutions quicker than all other algorithms on all benchmarks with the exception of the ILTI algorithm on the Mux11 benchmark. A significant improvement in run time was found for the Par08 benchmark.
The solution sizes produced by the NNGS algorithm are always larger than those found by the BP4A and ILTI algorithms with the exception of the Cmp06 results. The RDO scheme and ILTI algorithm both relay on traversing large tree libraries which make dealing with large bit problems very computationally intensive. As such, these methods do not scale well in comparison to the NNGS algorithm.
It is a clear that NNGS is weakest on the Mux11 benchmark. In this case a very large solution tree was found which consisted of 12,373 nodes. The multiplexer benchmark is significantly different from the other benchmarks by the fact that only four input arguments are significant to any single fitness case: the three addressing bits and the addressed bit. Perhaps this was the reason why the chosen methodology implemented in the controller resulted with poor results in this case.
8 Further Work
There are two possible branches of future research which stem from this work, the first centres around metaGP. As a deterministic set of rules, the proofofconcept controller is eminently suited to be encoded and evolved as part of a metaGP system. The difficulty in this case will be in appropriately choosing the set of operators which would be made available to the population of controller programs.
The second avenue of research which stems from this work involves encoding the current proofofconcept controller within the weights of a neural network (NN). This can be achieved through supervised learning in the first instance by producing training sets in the form of node triplets using the current controller. A training set would consist of randomly generated output arrays and the proofofconcept controller generated child output arrays. In this way, the actual Boolean problem solutions do not need to be found before training.
As part of the task of find a better controller, the weights of the NN could be evolved using genetic algorithms (GA), similar to the method employed by
[8]. The fitness of a NN weight set would correspond to the solution sizes obtained by the NNGS algorithm when employing the NN as a controller: the smaller the solutions, the better the weight set fitness. Using the proofofconcept controller in this way would ensure that the GA population would have a reasonably working individual within the initial population.A NN may also serve as a reasonable candidate controller for extending the NNGS algorithm to continuous symbolic regression problems. In this case, the input arguments of the problem would also form part of the NN’s input pattern.
Acknowledgments
I wish to thank Marc Schoenauer for interesting discussions and excellent supervision during my masters internship at INRIA, Paris, France where this work was conducted.
References
 [1] Koza, John R. Genetic programming: on the programming of computers by means of natural selection. Vol. 1. MIT press, 1992.
 [2] Koza, John R. ”Hierarchical Automatic Function Definition in Genetic Programming.” FOGA. 1992.

[3]
Krawiec, Krzysztof, and UnaMay O’Reilly. ”Behavioral programming: a broader and more detailed take on semantic GP.” Proceedings of the 2014 conference on Genetic and evolutionary computation. ACM, 2014.
 [4] Pawlak, T., Bartosz Wieloch, and Krzysztof Krawiec. ”Semantic backpropagation for designing search operators in genetic programming.” (2014): 11.
 [5] Krawiec, Krzysztof, and Tomasz Pawlak. ”Approximating geometric crossover by semantic backpropagation.” Proceedings of the 15th annual conference on Genetic and evolutionary computation. ACM, 2013.
 [6] Ffrancon, Robyn and Marc Schoenauer. ”Memetic Semantic Genetic Programming.” In Proceedings of the 2015 conference on Genetic and evolutionary computation. ACM, 2015 (in print).
 [7] Ffrancon, Robyn and Marc Schoenauer. ”Greedy Semantic Local Search for Small Solutions.” Semantic Methods in GP Workshop, GECCO Comp ’15: Proceedings of the 2015 Conference Companion on Genetic and Evolutionary Computation Companion. ACM, 2015 (in print).

[8]
Koutník, Jan, et al. ”Evolving largescale neural networks for visionbased reinforcement learning.” Proceedings of the 15th annual conference on Genetic and evolutionary computation. ACM, 2013.