DeepAI
Log In Sign Up

Retaining Experience and Growing Solutions

05/06/2015
by   Robyn Ffrancon, et al.
0

Generally, when genetic programming (GP) is used for function synthesis any valuable experience gained by the system is lost from one problem to the next, even when the problems are closely related. With the aim of developing a system which retains beneficial experience from problem to problem, this paper introduces the novel Node-by-Node Growth Solver (NNGS) algorithm which features a component, called the controller, which can be adapted and improved for use across a set of related problems. NNGS grows a single solution tree from root to leaves. Using semantic backpropagation and acting locally on each node in turn, the algorithm employs the controller to assign subsequent child nodes until a fully formed solution is generated. The aim of this paper is to pave a path towards the use of a neural network as the controller component and also, separately, towards the use of meta-GP as a mechanism for improving the controller component. A proof-of-concept controller is discussed which demonstrates the success and potential of the NNGS algorithm. In this case, the controller constitutes a set of hand written rules which can be used to deterministically and greedily solve standard Boolean function synthesis benchmarks. Even before employing machine learning to improve the controller, the algorithm vastly outperforms other well known recent algorithms on run times, maintains comparable solution sizes, and has a 100

READ FULL TEXT VIEW PDF
02/20/2018

Towards Deep Representation Learning with Genetic Programming

Genetic Programming (GP) is an evolutionary algorithm commonly used for ...
02/04/2005

Population Sizing for Genetic Programming Based Upon Decision Making

This paper derives a population sizing relationship for genetic programm...
01/19/2021

Controller Synthesis for Hyperproperties

We investigate the problem of controller synthesis for hyperproperties s...
06/25/2021

Learning Gradual Argumentation Frameworks using Genetic Algorithms

Gradual argumentation frameworks represent arguments and their relations...
12/06/2018

Formal Synthesis of Analytic Controllers for Sampled-Data Systems via Genetic Programming

This paper presents an automatic formal controller synthesis method for ...
05/10/2022

Scaling-up Generalized Planning as Heuristic Search with Landmarks

Landmarks are one of the most effective search heuristics for classical ...

1 Motivation

Most genetic programming (GP) [1] systems don’t adapt or improve from solving one problem to the next. Any experience which could have been gained by the system is usually completely forgotten when that same system is applied to a subsequent problem, the system effectively starts from scratch each time.

For instance, consider the successful application of a classical GP system to a standard n-bits Boolean function synthesis benchmark (such as the 6-bits comparator as described in [3]). The population which produced the solution tree is not useful for solving any other n-bits Boolean benchmark (such as the 6-bits multiplexer). Therefore, in general, an entirely new and different population must be generated and undergo evolution for each different problem. This occurs because the system components which adapt to solve the problem (a population of trees in the case of classical GP) become so specialised that they are not useful for any other problem.

This paper addresses this issue by introducing the Node-by-Node Growth Solver (NNGS) algorithm, which features a component called the controller, that can be improved from one problem to the next within a limited class of problems.

NNGS uses Semantic Backpropagation (SB)

and the controller, to grow a single S-expression solution tree starting from the root node. Acting locally at each node, the controller makes explicit use of the target output data and input arguments data to determine the properties (i.e. operator type or argument, and semantics) of the subsequently generated child nodes.

The proof-of-concept controller discussed in this paper constitutes a set of deterministic hand written rules and has been tested, as part of the NNGS algorithm, on several popular Boolean function synthesis benchmarks. This work aims to pave the way towards the use of a neural network as an adaptable controller and/or, separately, towards the use of meta-GP for improving the controller component. In effect, the aim is to exploit the advantages of black-box machine learning techniques to generate small and examinable program solutions.

The rest of this paper will proceed as follows: Section 2 outlines other related research. Section 3 details semantic backpropagation. A high level overview of the NNGS system is given in Section 4, and Section 5 describes the proof-of-concept controller. Section 6 details the experiments conducted. The experimental results and a discussion is given in Section 7. Section 8 concludes with a description of potential future work.

2 Related Work

The isolation of useful subprograms/sub-functions is a related research theme in GP. However, in most studies subprograms are not reused across different problems. In [2] for instance, the hierarchical automatic function definition technique was introduced so as to facilitate the development of useful sub-functions whilst solving a problem. Machine learning was employed in [3] to analyse the internal behaviour (semantics) of GP programs so as to automatically isolate potentially useful problem specific subprograms.

SB was used in [4] to define intermediate subtasks for GP. Two GP search operator were introduced which semantically searched a library of subtrees which could be used to solve the subtasks. Similar work was carried out in [6, 7], however in these cases subtree libraries were smaller and static, and only a single tree was iteratively modified as opposed to a population of trees.

3 Semantic Backpropagation (SB)

Semantic backpropagation (SB) within the context of GP is an active research topic [4, 5, 6, 7].

Consider the output array produced by the root node of a solution tree, where each element within the array corresponds to the output from one fitness case. This output array is the semantics of the root node. If the solution is perfectly correct, the output array will correspond exactly to the target output array of the problem at hand. In a programmatic style, the output array of a general node will be denoted as and the output from fitness case will be denoted by .

Each node within a tree produces an output array, a feature which has been exploited in [3] to isolate useful subtrees. The simplest example of a tree (beyond a single node) is a triplet of nodes: a parent node , the left child node , and the right child node .

As a two fitness case example, suppose that a triplet is composed of a parent node representing the operator , a left child node representing input argument , and a right child node representing input argument . The output array of the parent node is given by:

(1)

On the other hand, given the output array from the parent node of a triplet , it is possible to backpropagate the semantics so as to generate output arrays for the child nodes, if the reverse of the parent operator is defined carefully. This work will exclusively tackle function synthesis problems within the Boolean domain, and therefore, the standard [2, 3] set of Boolean operators will be used: , , , and .

1 1 1 0 0 # 0 # 0 # # # 1 1 # 1 # 1 0 0 0 # # # 1 0 # 1 # 0 0 1 1 # # # 1 0 0 0 1 # 0 # 1 # # #

Figure 1: Function tables for the reverse operators: , , , and .

Figure 1 gives function tables for the element-wise reverse operators: , , , and (their use with 1D arrays as input arguments follows as expected). As an example use of these operators the previous example will be worked in reverse: given the output array of , the arrays and are calculated as:

(2)
or

The hash symbol in this case indicates that either or will do. Note that two different possible values for and exist because . This feature occurs as a result of rows 4 and 5 of the function table. Note that each of the other reverse operators have similar features, specifically for: , , and .

Note also, that for any array loci in where , it is true that and . For example, .

Using the reverse operators in this way, output arrays can be assigned to the child nodes of any parent node. The child output arrays will depend on two decisions: Firstly, on the operator assigned to the parent node, as this is the operator that is reversed. And secondly, on the choices made (note the example above), at each loci, as to which of the two child output arrays contains the value. These decisions are made by the controller component.

Using these reverse operators for SB can only ever produce a pair of output arrays which are different from the problem target outputs in two ways. Firstly, the output arrays can be a flipped (using the gate on each bit) or an un-flipped versions of the problem target outputs. Secondly, some elements of the output arrays will be elements.

4 Node-by-Node Growth Solver (NNGS)

Figure 2: A visual representation of the NNGS algorithm during the development of a solution tree.

A visual representation of the NNGS algorithm can be seen in Fig. 2, which shows a snapshot of a partially generated solution tree. This tree, in it’s unfinished state, is composed of: and operators, an input argument labelled , and two unprocessed nodes. The basic role of the NNGS algorithm is to manage growing the solution tree by passing unprocessed nodes to the controller and substituting back the generated/returned node triplet.

1solution_tree {}
2.outputs target_outputs
3unprocessed_nodes {}
4
5while len(unprocessed_nodes) do
6      unprocessed_nodes.pop()
7 check for leaf node
8     if .type = ’argument’ then
9         solution_tree.insert()
10         continue move on to next node      
11
12      , , controller(, target_outputs)
13
14     unprocessed_nodes.insert({, })
15     solution_tree.insert()
16
17return solution_tree
Algorithm 1 The Node-by-Node Growth Solver
NNGS(target_outputs, controller)

Algorithm 1 gives a simple and more thorough explanation of NNGS. In line 2 the output values of the solution tree root node are set to the target output values of the problem at hand. The output values of a node are used, along with the reverse operators, by the controller (line 12) to generate the output values of the returned child nodes. The controller also sets the node type (they are either operators or input arguments) of the input parent node and generated child nodes.

Nodes which have been defined by the controller as input arguments (with labels: , , … etc.) can not have child nodes (they are by definition leaf nodes) and are therefore not processed further by the controller (line 8). When every branch of the tree ends in an input argument node, the algorithm halts.

Note that the controller may well generate a triplet where one or more of the child nodes require further processing. In this case the NNGS algorithm will pass these nodes back to the controller at a later stage before the algorithm ends. In effect, by using the controller component the NNGS algorithm simply writes out the solution tree.

5 Proof-Of-Concept Controller

Given an unprocessed node, the controller generates two child nodes and their output arrays using one of the four reverse operators. It also sets the operator type of the parent node to correspond with the chosen reverse operator that is used.

The ultimate goal of the controller is to assign an input argument to each generated child node. For example, suppose that the controller generates a child node with an output array and that an input argument is given by . In this case, can be assigned (can represent) the input argument because . The algorithm halts once each leaf node has been assigned an input argument.

Before giving a detailed description of the proof-of-concept controller, there are a few important general points to stress: Firstly, the entire decision making process is deterministic. Secondly, the decision making process is greedy (the perceived best move is taken at each opportunity). Thirdly, the controller does not know the location of the input node within the solution tree. The controller has priori knowledge of the input argument arrays, the operators, and the reverse operators only. Furthermore, the controller, in its current state, does not memorise the results of it’s past decision making. In this regard, when processing a node, the controller has knowledge of that node’s output array only. In this way, the controller acts locally on each node. Multiple instances of the controller could act in parallel by processing all unprocessed nodes simultaneously.

5.1 Step-by-step

Figure 3: Diagrammatic aid for the proof-of-concept controller.

This subsection will give a step-by-step run-through of the procedure undertaken by the proof-of-concept controller. Figure 3 serves as a diagrammatic aid for each major step.

5.1.1 Step 1

Given an input (parent) node , and for each reverse operator in Table 1, the first step taken by the controller is to generate output arrays for each of the child nodes. In the example given in step 1 of Fig. 3 only the reverse operator is used. The reverse operator generates values in the child output arrays due to the following property . In this step, whenever the opportunity arises (regardless of the reverse operator employed), all generated values within the child output arrays will be placed in the output array of the right child node . For example in the case of : will be used and not .

Note that the reverse operators propagate all elements from parent to child nodes. This feature is exemplified in step 1 of Fig. 3 by the propagation of the value at locus 4 of to loci 4 of both and .

5.1.2 Step 2

By this step, the controller has generated four different (in general) arrays, one for each reverse operator. The goal for this step is to compare each of those arrays to each of the possible input argument arrays (, … etc). As an example, in step 2 of Fig. 3 the generated array is compared to the input argument array.

Two separate counts are made, one for the number of erroneous 0 argument values and one for the number of erroneous 1 argument values (highlighted in blue and red respectively in Fig. 3). Two further counts are made of the number of erroneous loci, for 0 and 1 input arguments values, which could have been values (and therefore not erroneous) had the controller not specified in step 1 that all values should be placed in the array whenever possible. These last two statistics will be denoted by and for 0 and 1 input arguments values respectively. These four statistics form an error table for each reverse operator-input argument pair.

5.1.3 Step 3

In this step, the controller sorts the error tables by a number of statistics. Note that and are the number of remaining erroneous 0 argument values and erroneous 1 argument values respectively if all values were moved from the array to the array whenever possible. To simplify matters we note that

(3)
otherwise

Each error table is ranked by (in order, all increasing): , , , , and the number of values in . In a greedy fashion, the very best error table (lowest ranked) will be select for the next step (in Fig. 3 the error table is selected). Note that the ranked list of error tables might need to be revisited later from step 5.

5.1.4 Step 4

The error table selected in step 3 effectively serves as an instruction which details how the and
arrays should be modified. The goal of the controller is to move the minimum number of values from the array to the array such as to satisfy the error count for either ’s or ’s in one of the input arguments. In the example given in Fig. 3, two values in are swapped with 1’s in .

5.1.5 Step 5

In this step, the algorithm checks that the generated
and arrays do not exactly equal either the parent node or the grand parent node (if it exists). If this check fails, the algorithm reverts back to step 3 and chooses the next best error table.

5.1.6 Step 6

The final step of the algorithm is to appropriately set the operator type of given the final reverse operator that was used. In this step the algorithm also checks whether either (or both) of the child nodes can represent input arguments given their generated output arrays.

6 Experiments

The Boolean function synthesis benchmarks solved in this paper are standard benchmarks within GP research [2, 3, 4, 6]. They are namely: the comparator 6bits and 8bits (CmpXX), majority 6bits and 8bits (MajXX), multiplexer 6bits and 11bits (MuxXX), and even-parity 6bits, 8bit, 9bits, and 10bits (ParXX).

Their definitions are succinctly given in [3]:

“For an v-bit comparator Cmp v, a program is required to return true if the v/2 least significant input bits encode a number that is smaller than the number represented by the v/2 most significant bits. In case of the majority Maj v problems, true should be returned if more that half of the input variables are true. For the multiplexer Mul v, the state of the addressed input should be returned (6-bit multiplexer uses two inputs to address the remaining four inputs, 11-bit multiplexer uses three inputs to address the remaining eight inputs). In the parity Par v problems, true should be returned only for an odd number of true inputs.”

The even-parity benchmark is often reported as the most difficult benchmark [2].

7 Results and Discussion

Run time [seconds] Program size [nodes]
NNGS BP4A ILTI NNGS BP4A ILTI RDO
Cmp06 0.06 15 9 99 156 59 185
Cmp08 0.86 220 20 465 242 140 538
Maj06 0.19 36 10 271 280 71 123
Maj08 3.09 2019* 27 1391 563* 236 -
Mux06 0.21 10 9 333 117 47 215
Mux11 226.98 9780 100 12373 303 153 3063
Par06 0.35 233 17 515 356 435 1601
Par08 5.73 3792* 622 2593 581* 1972 -
Par09 25.11 - 5850 5709 - 4066 -
Par10 120.56 - - 12447 - - -
Table 1: Results for the NNGS algorithm when tested on the Boolean benchmarks, perfect solution were obtained for each run. BP4A columns are the results of the best performing algorithm from [3] (* indicates that not all runs found perfect solution). The RDO column is taken from the best performing (in terms of fitness) scheme in [4] (note that in this case, average success rates and average run times were not given).

The results are given in Table 1 and show that the NNGS algorithm finds solutions quicker than all other algorithms on all benchmarks with the exception of the ILTI algorithm on the Mux11 benchmark. A significant improvement in run time was found for the Par08 benchmark.

The solution sizes produced by the NNGS algorithm are always larger than those found by the BP4A and ILTI algorithms with the exception of the Cmp06 results. The RDO scheme and ILTI algorithm both relay on traversing large tree libraries which make dealing with large bit problems very computationally intensive. As such, these methods do not scale well in comparison to the NNGS algorithm.

It is a clear that NNGS is weakest on the Mux11 benchmark. In this case a very large solution tree was found which consisted of 12,373 nodes. The multiplexer benchmark is significantly different from the other benchmarks by the fact that only four input arguments are significant to any single fitness case: the three addressing bits and the addressed bit. Perhaps this was the reason why the chosen methodology implemented in the controller resulted with poor results in this case.

8 Further Work

There are two possible branches of future research which stem from this work, the first centres around meta-GP. As a deterministic set of rules, the proof-of-concept controller is eminently suited to be encoded and evolved as part of a meta-GP system. The difficulty in this case will be in appropriately choosing the set of operators which would be made available to the population of controller programs.

The second avenue of research which stems from this work involves encoding the current proof-of-concept controller within the weights of a neural network (NN). This can be achieved through supervised learning in the first instance by producing training sets in the form of node triplets using the current controller. A training set would consist of randomly generated output arrays and the proof-of-concept controller generated child output arrays. In this way, the actual Boolean problem solutions do not need to be found before training.

As part of the task of find a better controller, the weights of the NN could be evolved using genetic algorithms (GA), similar to the method employed by 

[8]. The fitness of a NN weight set would correspond to the solution sizes obtained by the NNGS algorithm when employing the NN as a controller: the smaller the solutions, the better the weight set fitness. Using the proof-of-concept controller in this way would ensure that the GA population would have a reasonably working individual within the initial population.

A NN may also serve as a reasonable candidate controller for extending the NNGS algorithm to continuous symbolic regression problems. In this case, the input arguments of the problem would also form part of the NN’s input pattern.

Acknowledgments

I wish to thank Marc Schoenauer for interesting discussions and excellent supervision during my masters internship at INRIA, Paris, France where this work was conducted.

References

  • [1] Koza, John R. Genetic programming: on the programming of computers by means of natural selection. Vol. 1. MIT press, 1992.
  • [2] Koza, John R. ”Hierarchical Automatic Function Definition in Genetic Programming.” FOGA. 1992.
  • [3]

    Krawiec, Krzysztof, and Una-May O’Reilly. ”Behavioral programming: a broader and more detailed take on semantic GP.” Proceedings of the 2014 conference on Genetic and evolutionary computation. ACM, 2014.

  • [4] Pawlak, T., Bartosz Wieloch, and Krzysztof Krawiec. ”Semantic backpropagation for designing search operators in genetic programming.” (2014): 1-1.
  • [5] Krawiec, Krzysztof, and Tomasz Pawlak. ”Approximating geometric crossover by semantic backpropagation.” Proceedings of the 15th annual conference on Genetic and evolutionary computation. ACM, 2013.
  • [6] Ffrancon, Robyn and Marc Schoenauer. ”Memetic Semantic Genetic Programming.” In Proceedings of the 2015 conference on Genetic and evolutionary computation. ACM, 2015 (in print).
  • [7] Ffrancon, Robyn and Marc Schoenauer. ”Greedy Semantic Local Search for Small Solutions.” Semantic Methods in GP Workshop, GECCO Comp ’15: Proceedings of the 2015 Conference Companion on Genetic and Evolutionary Computation Companion. ACM, 2015 (in print).
  • [8]

    Koutník, Jan, et al. ”Evolving large-scale neural networks for vision-based reinforcement learning.” Proceedings of the 15th annual conference on Genetic and evolutionary computation. ACM, 2013.