Semantically-Oriented Mutation Operator in Cartesian Genetic Programming for Evolutionary Circuit Design

04/23/2020 ∙ by David Hodan, et al. ∙ Brno University of Technology 0

Despite many successful applications, Cartesian Genetic Programming (CGP) suffers from limited scalability, especially when used for evolutionary circuit design. Considering the multiplier design problem, for example, the 5x5-bit multiplier represents the most complex circuit evolved from a randomly generated initial population. The efficiency of CGP highly depends on the performance of the point mutation operator, however, this operator is purely stochastic. This contrasts with the recent developments in Genetic Programming (GP), where advanced informed approaches such as semantic-aware operators are incorporated to improve the search space exploration capability of GP. In this paper, we propose a semantically-oriented mutation operator (SOMO) suitable for the evolutionary design of combinational circuits. SOMO uses semantics to determine the best value for each mutated gene. Compared to the common CGP and its variants as well as the recent versions of Semantic GP, the proposed method converges on common Boolean benchmarks substantially faster while keeping the phenotype size relatively small. The successfully evolved instances presented in this paper include 10-bit parity, 10+10-bit adder and 5x5-bit multiplier. The most complex circuits were evolved in less than one hour with a single-thread implementation running on a common CPU.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

The design of combinational logic circuits represents a popular research topic addressed by the evolutionary computing community since the early nineties. The first efforts for evolving various combinational circuits were made by Koza (Koza, 1992), Coello et al (Coello et al., 1998) and Miller et al (Miller et al., 1998)

. Koza investigated the evolutionary design of the even-parity problem in his extensive study related to Genetic Programming (GP). Coello et al. employed the same algorithm to evolve 2-bit adders and multipliers. In contrast to these studies, Miller used a variant of genetic algorithm and successfully evolved more complex circuits such as 4-bit adders and 3-bit multipliers. But the evolutionary design of logic circuits has attracted more attention of researchers with the advent of a new form of genetic programming called Cartesian Genetic Programming (CGP) proposed in 2000 

(Miller and Thomson, 2000a).

Since its introduction, CGP has been adopted by many researchers and successfully applied to many application areas including optimization of Boolean circuits (Miller and Thomson, 2000b; Miller, 2019; Vasicek, 2015). Despite its success, CGP suffers from limited scalability, especially when used for evolutionary circuit design (i.e. when evolving Boolean circuits from scratch) (Vasicek, 2017). To address this issue, researchers introduced various strategies (Vasicek and Sekanina, 2014; Goldman and Punch, 2015; da Silva et al., 2019). Nevertheless, the most complex circuit evolved directly (i.e., without introducing a decomposition) from a randomly generated initial population is a 28-input benchmark (Vasicek and Sekanina, 2014). Considering the common arithmetic circuits, 55-bit multiplier and 9+9-bit adder represent the largest problem instances evolved from scratch (Hrbacek and Sekanina, 2014). In fact, the evolutionary design of multipliers represents a tough problem even for small bit widths. A multi-thread parallel CGP accelerator running for 4 hours on a cluster of 60 computers was used by Hrbacek et al. (Hrbacek and Sekanina, 2014) to obtain a single 55-bit multiplier.

Crossover and mutation operators are typically used to create offspring in standard GP. In contrast to that, CGP was designed and is still used with a mutation operator representing the driving force of the evolution (Miller, 2019). Despite several attempts to introduce a crossover operator to CGP (Clegg et al., 2007; Slany and Sekanina, 2007; da Silva and Bernardino, 2018), the standard CGP with mutation operator remains the best strategy, especially when considering the design of logic circuits. One of the possible explanations of the mutation operator’s superiority over crossover is that the Boolean domain implies a challenging search space, especially for non-trivial circuit structures such as multipliers (Miller et al., 2000; Sekanina, 2017).

The point and probabilistic mutation are the most used forms of mutation in standard CGP (Miller, 2019). However, there is a considerable length and positional bias associated with the usage of these operators (Miller, 2019). Moreover, the efficiency of the search deteriorates with increasing the problem size due to the presence of inactive gates. Since many genes in CGP are redundant, mutations often occur only in the redundant regions, which means that the mutated genotype has the same phenotype as its parent. Various improvements have been proposed to increase the efficiency of the CGP. For example, Single Active Mutation (SAM) operator (Goldman and Punch, 2013a) was proposed to reduce the wasted objective function evaluations. SAM ensures that in addition to some inactive genes, exactly one active gene is mutated. Moreover, different strategies have been proposed to compensate the positional bias (Miller, 2019). Despite these improvements, the mutation operator remains still blind in the sense that mutated genes, as well as the value of the mutated genes, are chosen randomly. This contrasts with the recent development in GP, where the researchers try to incorporate more advanced informed approaches such as semantic-aware operators to improve the capability of exploring the search space (Ffrancon and Schoenauer, 2015; Moraglio and Krawiec, 2017; Pawlak and Krawiec, 2018). According to our best knowledge, the only approach employing a kind of informed (or biased) mutation operator in CGP is the work of Silva et al. (da Silva et al., 2019). The authors introduced a guided active mutation (GAM) which aims to reduce the number of evaluations needed to find feasible solutions. GAM consists of modifying one or more active nodes on the subcircuit from the inputs to the output with the smallest number of correct values when compared to the truth table. It means that only genes associated with the active nodes are mutated. The mutated genes and their values are chosen randomly. GAM does not mutate the subgraphs of outputs that already produce a valid response according to the truth table. When evaluated on Boolean problems, however, GAM achieved low success rates when compared to SAM, and it seems to be necessary to combine both strategies together (da Silva et al., 2019). For single-output problems, it degrades to SAM.

This paper proposes a new mutation operator aiming to reduce the number of evaluations needed to design fully working combinational circuits. The semantic is used to guide the mutation operator and avoid wasted evaluations. The point mutation in CGP replaces the mutated genes with randomly chosen (but valid) values. When a gene associated with input connection is mutated in our case, we try to compute the optimal connection. Moreover, a different seeding strategy is used in our case. We always start with a candidate solution consisting of inactive nodes only. Similarly to SAM, only genes associated with the active nodes and program outputs are mutated. The efficiency of the proposed method is evaluated on standard benchmark problems such as parity circuits and adders as well as on hard benchmark problems, which is the design of combinational multipliers in which the fitness in CGP typically stagnates for many generations without any improvement. The method is compared to the best CGP-based and GP-based methods available in the literature.

2. Relevant Work

2.1. Semantic Genetic Programming

GP operators traditionally work in the syntactic space and manipulate the syntax of parents. The parents can also be modified based on their semantics. The semantics of a program can be formally defined in a number of ways. It can be a canonical representation, a description of the behavior based on a logical formalism, or a set of input-output pairs making up the computed function (Moraglio et al., 2012). In the latter case, we can sample the inputs at random, or we can enumerate all possible input combinations. The exhaustive enumeration is typical, especially for Boolean problems.

Semantic GP is a branch of GP that involves semantic information to improve various aspects of GP. The semantic, for example, can be used to enforce semantic diversity during the evolutionary process (Moraglio et al., 2012). Moreover, we can use semantics to boost the performance of the search space exploration by avoiding returning to points that have already been traversed. Another possibility is to prevent genetic operators from causing a destructive change in fitness(Beadle and Johnson, 2009).

In 2012, Moraglio et al proposed Geometric Semantic Genetic Programming (GSGP) which uses specific genetic operators, the so-called geometric semantic operators (Moraglio et al., 2012)

. On many various symbolic regression and classification problems, it has been shown that GSGP provides statistically better results than a common genetic programming and other machine learning methods in terms of the error score 

(Martins et al., 2018). The reason is that by applying these operators, one can effectively create a unimodal error surface for problems such as symbolic regression. The search process conducted in such a search space is more efficient than in the case of a standard GP. However, geometric semantic operators, by construction, always produce offspring that are larger than their parents, causing a fast growth in the size of the individuals. The growth is linear for mutation and exponential for crossover (Moraglio et al., 2012). As a consequence of that, the evolved programs undergo unprecedented growth in their size. This leads to excessive usage of memory and computational power, and also results in non-interpretable solutions (Castelli et al., 2015).

Several approaches addressing the problem of the exponential growth of GSGP individuals have been developed, but the problem is still considered unsolved. One branch of the methods is based on simplifying the offspring during the evolution. The other approach is to find new and better versions of the semantic crossover operators (Moraglio et al., 2012; Martins et al., 2018). Another possibility is making the algorithm more efficient in terms of memory and computational resources (Castelli et al., 2015).

2.2. Cartesian Genetic Programming

CGP grew from a method of evolving digital circuits developed by Miller et al. in 1998 (Miller et al., 1998) to address two issues related to the efficiency of common GP – poor ability to represent digital circuits, and the presence of bloat. Compared to tree-based GP, CGP represents the problems using directed acyclic graphs (DAGs) encoded using fixed length structures called netlists (i.e., one-dimensional or two-dimensional grid of nodes). This representation has many advantages (Miller, 2019). The graphs can represent many types of computational structures with arbitrary number of outputs. The nodes can be used multiply to create more complex blocks. The fixed-length netlists can contain non-coding (inactive) genes allowing the presence of variable-length phenotypes.

Figure 1. Example of a CGP individual representing a 2-bit multiplier having four inputs and four outputs encoded by one-dimensional CGP with parameters: , , , , .

2.2.1. Circuit Representation

A two-dimensional array of computational nodes arranged in columns and rows is used to represent the programs (individuals). CGP utilizes the following encoding scheme. The program inputs and node outputs are labeled and , respectively, where is the number of program inputs. Each node input can be connected either to the output of a node placed in previous columns or to one of the program inputs. This avoids feedback which is not desirable for combinational circuits. A candidate solution consisting of two-input computational nodes is represented in the chromosome by triplets consisting of three genes determining for each node its function , and label of nodes and to which its inputs are connected. The last part of the chromosome contains genes specifying the labels of nodes or program inputs where the program outputs are connected to.

Depending on the function of a node, some of its inputs may become redundant. For example, a two-input computational node that implements inverter does effectively utilize only one input. Moreover, some of the nodes may become redundant because they are not directly or indirectly connected to a program output. These nodes are typically referred to as inactive nodes. The presence of inactive nodes enables the existence of neutral mutations. According to many studies, this neutrality is important for an effective search in CGP (Vassilev and Miller, 2000). Figure 1 demonstrates the principle of CGP encoding. It can be seen that although eight computational nodes are available in total, not all the nodes have to be employed in the phenotype (resulting circuit). The computational node is inactive so three corresponding genes can be mutated without affecting the phenotype.

2.2.2. Search Strategy

CGP employs a () evolutionary strategy (Miller and Thomson, 2000b). The algorithm is summarized in Algorithm 1 and consists of the following steps. The initial population of the size is generated randomly. Then, the following steps are repeated until the termination condition is satisfied: (i) the population is evaluated, (ii) a new parent is selected, (iii) offspring are created from the parent using mutation operator.

CGP utilizes a point mutation that modifies up to randomly chosen genes. The value of a selected gene is replaced with a randomly generated new one. Note that only certain values can be assigned to each gene to avoid feedback. The range of valid values depends on the gene position and can be determined in advance. It is worthwhile to mention that the previous parent is never selected by SelectFittestIndividual procedure as the new parent if there exists at least one individual who obtained the same or better fitness. This strategy is important because it ensures diversity between populations in different generations (Miller and Thomson, 2000b).

When considering an evolution from scratch, the fitness of a candidate solution is typically determined as follows. All the requested assignments are applied to the program inputs, and the number of bits that the candidate solution computes correctly corresponds to the fitness value (additional criteria can be incorporated as well). In case of circuit evolution, this procedure is, in fact, the computation of the Hamming distance (HD) between the specification given in the form of Truth table (TT) and the response of a candidate circuit :

The algorithm is terminated when the maximum number of generations is exhausted, or a required solution is obtained, i.e., when . Note that .

1create initial population ; ;
2 while terminating condition not satisfied do
3       EvaluatePopulation();
4       SelectFittestIndividual();
5       if fitness() fitness() then ;
6       ;
7      
8 end while
9return ;
Algorithm 1 Standard -CGP algorithm

3. Proposed Approach

3.1. Semantically-Oriented Mutation Operator

In this work, we propose to replace the purely stochastic mutation operators used in CGP with semantically-oriented mutation operator (SOMO). This work aims to improve the efficiency of the evolutionary design of combinational circuits. The principle of the proposed operator which replaces the Mutate procedure of Algorithm 1 is summarized in Algorithm 2.

The algorithm accepts the parental solution and produces its mutated version . The mutation operator is operating in phenotype space, hence the first step consists of decoding and obtaining a DAG formed by a set of nodes denoted as and a set of edges denoted as . Every CGP computational node is included in independently, whether it is active or not. Besides, the program inputs, as well as the program outputs, are treated as nodes. To distinguish between program inputs, program outputs, active, and inactive nodes, the set of nodes forming the program inputs and program outputs are provided. Moreover, a mapping assigning a function from to each node is produced. In the next step (line 2), active nodes are identified. The active nodes are those that are directly or indirectly connected to program output. Then one of the internal active nodes is chosen randomly for mutation. The mutated node is denoted as . The mutation can affect either a node function (i.e., ) or a node connection (i.e., ). Note that only node connection is mutated for a node from

. The function of an internal node is mutated with probability

. Otherwise, the connection of a single input is mutated. Finally, the modified DAG is encoded using CGP encoding scheme. The encoding works as follows. All the nodes are topologically sorted. The unchanged nodes preceding the mutated node are placed at the beginning of the chromosome, followed by the nodes that have been originally inactive. Finally, the nodes following together with are encoded. This arrangement is not, in fact, necessary from the theoretical point of view. Still, it simplifies the implementation of the mutation operator because we can use the advantage of the CGP encoding and use the connections based on the node indices (i.e., a node can be connected only to those nodes having the lower index to produce acyclic graphs.

Input: A CGP individual consisting of nodes
Output: A mutated individual
;
  // decode as a DAG with leaves and roots (outputs);
;
  // get active nodes
1 ;
2 if  then // mutate node function
3      
4else // mutate node connection
5       change connection and function of inactive nodes;
6       ;
7       ;
8       ;
9      
10 end if
return
Algorithm 2 Semantically-oriented mutation operator

The mutation of a node input connection consists of several steps. First, some inactive nodes are modified (the amount is defined as ratio ). The modification of a node includes the reconnection of all node inputs causing update of and the change of node function causing update of . The node can be connected to any active node preceding or any inactive node. New node function is chosen randomly from . This step ensures that new genetic material is generated before performing the actual mutation. As a consequence of that, several sub-circuits with provisionally unconnected outputs can arise during this step. After that, one input of a mutated node is chosen and reconnected to a node identified as the most suitable node, i.e., a node whose connection to causes improvement of the fitness value.

The identification of the most suitable node is based on semantics and is formally defined in Algorithm 3. The procedure uses the set of input-output pairs making up the computed function and calculates score for every node of DAG that may potentially be connected to the mutated node . Then, the highest-score node is returned. If more nodes receive the same score, the node closest to the program inputs is preferred. The score reflects the Hamming distance between values expected at the selected input of the mutated node and output values computed by a particular node. To determine the distance, the algorithm needs to know the set of input-output pairs and outputs for every node, which is obtained by simulating the DAG. We propose to perform the simulation for every possible input combination, which is formally defined as , but the computation of the score can be in general based on a subset of all possible input combinations.

Input: DAG , node function assignment , selected node and its input , specification in the form of a Truth table , where ,
Output: The most suitable node
1 initialize score(n) to 0 for every ;
;
  // get candidates for connection
2 foreach  do // perform simulation for all inputs
3       evaluate for input ;
4       evaluate for input and forced to ’0’ ;
5       evaluate for input and forced to ’1’ ;
       // determine desired input value for each output
       // update score of each node
6       foreach  do
7             ;
8            
9       end foreach
10      
11 end foreach
12return ;
Algorithm 3 Procedure identifyBestNode

Algorithm 3 starts with determining the set of candidate nodes . This set includes nodes that are not directly or indirectly connected to and whose connection thus does not cause a cycle. Then, DAG is simulated for every input combination. A particular input combination is applied at the program inputs (i.e., the outputs of DAG leaves ) and value at the output of every node is determined. The simulation is divided into three parts. Nodes included in are simulated at first. Then, the remaining nodes are simulated with the knowledge of the outputs at nodes included in for two different cases. The first case reflects the situation when the mutated input is forced to logic zero, and the second one the situation when is equal to logic one. This arrangement helps us to investigate which value the input should take for a particular combination at program inputs (denoted as ) to achieve a match between the value at the program outputs (denoted as or ) and the specification given in form of a Truth table (denoted as ). Note that the term in superscript points to a Boolean value associated with a program output node . The desired input value is denoted as and it can be equal to ‘0‘, ‘1‘ or ‘X‘, where ‘X‘ means that it does not matter what Boolean value the input takes. This can happen in two cases. One situation is that neither ‘0‘ nor ‘1‘ presented at leads to expected output response that matches the specification. The another situation that can happen is that both input values lead to required response and it does not matter which one will be chosen. To determine the value of , we use ternary operator and reduction operator defined as follows:

Figure 2. Principle of the most suitable node identification (part b) when mutating a DAG consisting of six computational nodes (part a) given a specification defining the expected output values and mutated node whose second input is going to be changed. At the end of the process, node is determined as the most suitable candidate.

For a particular program output, the operator takes the required output value according to TT together with the output values obtained when and and determines what input produces the required output. The reduction operator is used to combine the identified values from multiple program outputs to a single value denoted as . Finally, the score of each candidate node is updated. In this step, the algorithm compares the value at the node output which can be either ‘0‘ or ‘1‘ with and increments the score provided that a match is detected. The increment corresponds with the inverted Hamming distance which is defined as follows:

The whole process is illustrated in Figure 2 on a simple problem having two program inputs and single program output. The DAG consists of six nodes denoted as . Node is the mutated node whose second input will be reconnected. Node and are not active and were produced at line 2 of Algorithm 2. We simulate the left part of the DAG denoted as for every input combination, i.e., . The right part is divided into two separate cases as described above and simulated too. At the end of the simulation, we receive four responses at the output of every node in . For node , for example, we obtained these values: for the inputs in . The specification requires the combination at the program output corresponding to the node . We obtained the response and when is forced to ‘0‘ and ‘1‘, respectively. Looking at the rightmost value, we can see that the specification requires ‘1‘, which is available when . The next required value is available when and the third when . In the last, i.e. leftmost, case neither nor provide ‘0‘ at the output, hence the corresponding position of will be equal to ‘X‘. The complete content of is equal to and forms a kind of pattern which needs to be compared with outputs of all nodes that can potentially be connected at the input of . The highest score is received for because three out of four values corresponds to the pattern. Therefore, is returned as the result of the procedure identifyBestNode.

3.2. Population Initialization

Compared to the standard CGP, we propose to use a different strategy for population initialization. We hypothesize that it is better to start with a candidate solution having no active gate to maximize the efficiency of the proposed approach and minimize the number of active gates of the evolved solutions. Therefore, the initial population consists of individuals whose program outputs are connected directly to one of the program inputs. The selection of the inputs is done randomly. As the program outputs are treated as nodes in Algorithm 2, the mutation operator naturally selects one of the output nodes in the first generations.

3.3. Properties

3.3.1. Linear size-dependent overhead against CGP

Similarly to SAM (Goldman and Punch, 2013a), mostly adopted in CGP, a single active node is always mutated. The mutation of active nodes helps to reduce the wasted evaluations and improves the efficiency of the search. Considering this fact, the efficiency of the search performed by SOMO expressed in terms of the number of generations needed to find a fully working solution needs to be at least the same as in CGP with SAM. However, the computational complexity required to create a single generation is higher in SOMO. The overhead compared to the CGP is linear w.r.t. the number of nodes and can be controlled by , which determines together with the number of active nodes the total number of simulated nodes. Compared to CGP, we need to simulate a part of the circuit twice. In addition, comparison with values at node outputs is needed in SOMO to calculate score of each node (line 3 in Algorithm 3).

3.3.2. Absence of uncontrollable growth in the size

As discussed in Section 2.1, semantic GP suffers from fast growth in the size of the individuals because the semantic operators always produce offspring that are larger than their parents. In our case, the growth in the size is linear in the worst case (a constant number of nodes can be activated in each generation). But the main feature of SOMO is that it can deactivate already active nodes in the course of the evolution. Moreover, some other advantages of CGP encoding have been implicitly inherited. For example, CGP uses a limited number of computational nodes and naturally does not suffer from a phenomenon called bloat (Miller, 2019).

3.3.3. Compensation of positional and length bias

It has been shown that CGP is naturally biased towards phenotypes of a given size (Goldman and Punch, 2013b, 2015). Typically a small percentage of the available nodes is utilized. In addition, there is a strong positional bias in CGP causing an increase in the likelihood that nodes close to the inputs will be active (Miller, 2019). To compensate both these effects, Reorder operator was introduced by Goldman et al. (Goldman and Punch, 2015). Both phenomena are naturally mitigated in SOMO. Node reordering is, in fact, performed by the Encode procedure (line 2 in Alg. 2). Moreover, the number of active nodes may increase naturally due to the construction of SOMO.

4. Experimental Evaluation

4.1. Experimental Setup

The proposed semantically-oriented operator was implemented in C++ and integrated within a standard CGP algorithm. The application is implemented as a single thread code. We adopted the principle of parallel simulation introduced in (Vasicek and Slany, 2012) to maximize the performance of the circuit simulation. Following Hrbacek et al. (Hrbacek and Sekanina, 2014), each candidate solution is transformed into native 256-bit AVX2 instructions. This arrangement helps us to evaluate circuits having up to 8 inputs in a single pass through the CGP nodes.

The proposed method is evaluated on the evolutionary design of adders, multipliers, and parity circuits. Although the construction of an optimal parity circuit is a straightforward process, parity circuits are considered to be an appropriate benchmark problem within the evolutionary computation community. Usually, a small set of gates (and, or, not) is used. But even if we allow the usage of the xor gates, the design of parity circuits is hard due to the presence of a deceptive landscape, especially for higher bit widths (Vasicek, 2017)

. The evolutionary design of multipliers represents a hard problem due to the complexity of the multiplication (the multipliers consist of a sequence of adders reducing the partial products to a single output vector).

The evolutionary algorithm uses population of

individuals where . This setting enables us to investigate exclusively the impact of the proposed mutation operator. In each mutation, all inactive nodes are randomly changed, i.e. . This design choice allows us to investigate the rate of growth in the size. Only the connections are modified, i.e. . The function set consists of common binary logic gates

not, and, or, xor, nand, nor, xnor

. To provide a fair comparison with the literature, the parity circuits are also evaluated with a reduced function set not, and, or, nand, nor.

As recommended in the literature, we use the one-dimensional form of CGP array consisting of a single row of computational nodes. The number of rows is fixed to 1. The number of columns needs to be chosen according to the complexity of the addressed problem. Moreover, the standard CGP typically works better under the presence of a reasonable degree of redundancy (Miller, 2019), i.e. larger genotypes are needed compared to the minimum number of logic gates required to implement a fully functional circuit. Table 1 reports the number of standard gates denoted as required to implement ripple-carry adder, array multiplier, and parity circuit. The number of the available CGP nodes is derived from this parameter as .

Adder 2+2 3+3 4+4 5+5 6+6 7+7 8+8 9+9 10+10
4 6 8 10 12 14 16 18 20
3 4 5 6 7 8 9 10 11
7 12 17 22 27 32 37 42 47
Multiplier 22 33 44 55
4 6 8 10
4 6 8 10
11 33 67 113
Parity 4 5 6 7 8 9 10
4 5 6 7 8 9 10
1 1 1 1 1 1 1
9 12 15 18 21 24 27
Table 1. The minimal number of gates required to implement the selected benchmark problems for a given number of inputs . Parameter is the number of outputs. For parity circuits, the number of and/or/not gates is reported.

In total, 270 configurations (defined by circuit, chromosome size and function set /) are analyzed and for each configuration, 15 independent runs are executed. The search is terminated either when a fully functional solution is evolved or when the 12-hour limit is elapsed. As a consequence of the chosen parameter setting (, ), there is a high chance that the search gets stuck in a local optimum when all the available nodes are active. Therefore, we implemented an additional termination condition to avoid wasted CPU time. After 15 minutes without any progress, the algorithm is aborted with the result counted as unsuccessful. All experiments are conducted on Intel® CPU E5-2630 @ 2.20 GHz.

For each run, it is evaluated how long it takes to find a fully functional solution exhibiting (denoted as execution time), how many generations are needed (denoted as # generations) and how many nodes are active in the discovered solution (denoted as active nodes). For each configuration we calculated computational effort as defined in (Koza, 1992; Miller and Thomson, 2000b) and success rate (the proportion of runs where a fully functional circuit is found). The computational effort is calculated for .

Figure 3. The median execution time needed to find fully functional circuits, the median success rate and the median circuit size for various setting of .

4.2. Role of the Available Number of Nodes

First, we analyze the impact of the amount of available CGP nodes on the efficiency of the search process. Three aspects are investigated – the number of generations and the execution time required to evolve a fully functional solution, size of the evolved solutions and the success rate. The statistics calculated from all 15 runs are shown in Figure 3. To give a better idea about the real computational complexity, we report the average time needed to find a fully functional solution in this figure. Due to the limited space, only two instances of each benchmark problem are presented.

As can be seen, the success rate increases with increasing . The results show that it is necessary to introduce some degree of redundancy in . If is equal to , it is hard to find any solution, and the success rate is very low (see 3-bit multiplier) or even zero (adders, 5-bit multiplier, parity circuits). This is caused by the fact that all the available nodes are active. It is thus hard to make any progress because .

A typical dependency of the execution time on is visible for the 3-bit multiplier. If is close to , many generations are needed to find a solution. The number of generations decreases with increasing success rate and then remains relatively constant. This effect is visible in Figure 5. The problem is, however, that the execution time increases with increasing because more nodes need to be simulated in each generation. Let us recall that all nodes may be modified (and thus simulated) in every mutation due to the setting of parameter. This setting was chosen intentionally to see this effect. In a real scenario, it is better to mutate a constant amount of non-active nodes. According to the statistical evaluation, around one hundred nodes were activated at most during a single mutation.

The circuit size grows with the increasing number of available nodes. Despite the exponential growth in , the growth in the size is quite linear. Surprisingly, the evolved circuits are relatively compact even though we did not implement any explicit mechanism which forces the search towards more compact solutions. For 10-bit parity and , for example, we obtained circuits with less than 40 gates even though the number of available nodes is 135.

Figure 4. The convergence curves for 5-bit multiplier design and given configuration of . The median fitness value is reported. A fully functional circuit has zero fitness.

The evolution of the 5-bit multipliers exhibits the worst success rate. The first four configurations of never reached a solution. The search typically got stuck at local optima for a longer period and it was prematurely terminated due to the presence of a hard time limit. When we removed the limit, we were able to evolve multipliers for . Despite that, no multiplier was evolved for lower values of . The explanation can be seen when we look at the results shown in Figure 5. Around 60 times more generations are needed to evolve 3-bit multiplier for compared to the setting . It means that the insufficient number of generations is provided for 5-bit multipliers. Figure 4 shows the typical convergence curves for all configurations. The higher , the faster convergence.

Minimum, maximum and the median number of generations needed to find a fully functional 3-bit and 5-bit multiplier is analysed in Figure 5. We can see that the performance of the proposed method is relatively robust regarding parameter setting. This property is visible on 3-bit multipliers and is also observable on adders and parity circuits where we can see a relative low sensitivity of the computational complexity to the chosen value .

Figure 5. The number of generations required to find a fully functional 3-bit and 5-bit multiplier.
Circuit Succ. Comp. Execution time [s] # Generations (evaluations) # Active nodes (circuit size)
rate effort min mean max min mean max min mean max
Adder 2+2 4 3 350 100% 365 0.01 0.04 0.07 66 166.9 365 36 64.5 102
4+4 8 5 85 100% 5,972 0.16 0.44 1.12 830 2,376 5,972 32 41.2 48
6+6 12 7 270 100% 20,461 0.64 2.25 4.93 2,381 8,466 20,461 73 97.7 150
8+8 16 9 185 100% 29,922 5.39 20.82 37.36 4,074 13,872 29,922 76 85.9 102
9+9 18 10 210 100% 31,206 60.69 169.99 412.08 6,041 14,801 31,206 72 90.8 114
10+10 20 11 235 100% 42,739 532.77 1,326 2,384 9,300 23,544 42,739 78 101.8 124
Mult. 2x2 4 4 220 100% 287 0.01 0.03 0.06 58 164.0 287 33 52.3 64
3x3 6 6 660 100% 15,134 0.33 1.26 3.21 1,576 5,977 15,134 141 196.8 244
4x4 8 8 3,350 100% 188,710 11.83 38.59 97.21 28,743 85,253 188,710 773 1,049 1,328
5x5 10 10 11,300 27% 742,497 1,278 2,556 3,493 387,860 712,077 1,090,909 3,671 4,476 5,189
Parity 5 5 1 60 100% 147 0.002 0.009 0.03 5 38.9 147 6 18.1 30
6 6 1 75 100% 235 0.005 0.02 0.04 23 87.1 235 11 23.7 51
7 7 1 90 100% 450 0.007 0.04 0.07 28 191.8 450 12 28.1 50
8 8 1 105 100% 705 0.007 0.04 0.13 33 185.9 705 13 33.7 53
9 9 1 240 100% 1,029 0.009 0.05 0.19 43 280.5 1,029 47 71.1 119
10 10 1 270 100% 1,794 0.02 0.12 0.35 77 596.5 1,794 43 79.6 125
Parity 5 5 1 240 100% 1,636 0.06 0.13 0.34 264 616.1 1,636 61 70.6 86
6 6 1 300 100% 2,991 0.13 0.28 0.56 591 1,376 2,991 52 84.1 131
7 7 1 180 100% 9,406 0.19 0.61 1.92 1,019 3,132 9,406 55 74.9 100
8 8 1 210 100% 13,078 0.38 1.13 2.42 2,045 5,772 13,078 58 85.0 109
9 9 1 240 100% 14,102 0.61 1.59 2.65 2,872 7,793 14,102 78 106.5 151
10 10 1 135 100% 18,950 0.67 2.36 3.71 3,544 11,647 18,950 63 78.1 98
Table 2. Overall statistics computed for a configuration exhibiting the best mean time required to find a working solution

4.3. Overall results

The overall results are summarized in Table 2. For each circuit, the statistics related to the execution time, required number of evaluations, and the number of nodes is provided for such that achieves the best mean search time. The minimal, maximal, and mean values are reported together with the 0.95 confidence level.

We can see that the execution time increases exponentially with the increasing bit width, however, it is still acceptable even for the largest circuits. The number of active nodes is enormous considering the conventional circuit implementations, but the circuit size is not subject of the evolution. For parity circuits, we report the results for the complete as well as reduced function set. The reduced function set causes a increase in the execution time and produces approximately larger circuits.

4.4. Comparison with the literature

4.4.1. Evolutionary circuit design using CGP

The 6-bit adders were successfully evolved using an advanced mutation operator GAM in (da Silva et al., 2019). More than evaluations were needed for the best configuration (1+3) producing circuits having from 116 to 175 gates. SOMO requires 127x fewer evaluations on average and produces more compact circuits (73 - 150 gates). As discussed in Section 3.3.1, a single SOMO evaluation corresponds up to two truth table evaluations in common CGP. Considering this fact, SOMO still performs substantially better than GAM.

The computational effort of different CGP mutation and crossover operators is evaluated in (da Silva and Bernardino, 2018). Although many powerful techniques improving CGP for digital circuit design have been proposed, the SOMO achieves 30 - 114x lower computational effort compared to the best method (see Table 3). Note that adders with carry input are considered in this comparison.

Circuit CGP MC-CGP MC-ECGP X-CGP Our Reduction
Adder 3+2 496,200 140,800 24,200 54,661 798
Adder 4+3 8,190,400 1,286,000 1,230,400 360,746 3,171
Mult. 2x2 52,000 11,200 22,400 32,962 287
Mult. 3x3 18,509,600 873,600 867,600 950,374 15,134
Table 3. Computational effort of different variants of CGP (results for CGP, MC-CGP, MC-ECGP, and X-CGP taken from (da Silva and Bernardino, 2018))

The most complex arithmetic circuits evolved using CGP were reported by Hrbacek et al. (Hrbacek and Sekanina, 2014). Their multi-threaded parallel implementation of CGP discovered a 5-bit multiplier on a supercomputer cluster in 548 core-hours. Due to high computational requirements, only a single run was executed. SOMO can design this multiplier in 42 minutes on average (i.e., 771x faster) on a standard CPU. The 9-bit adders were discovered in 2.3 core-hours compared to the average 170 seconds (50x speedup) needed by SOMO.

4.4.2. Evolutionary circuit design using Semantic GP

Ffrancon et al. reports results for parity circuits with the reduced function set  (Ffrancon and Schoenauer, 2015). The authors were able to design 6-, 8-, and 9-bit parity in 164, 622, and 5850 seconds on average, respectively. SOMO accelerates the design process 119x, 586x, and 5098x, respectively. The experiments were conducted on a comparable CPU. However, it is fair to say that the authors implemented the algorithm in Python. The obtained circuits were quite bulky (435, 1972, and 4066 gates) compared to SOMO (75, 106 and 75 gates).

Parity circuits were also evolved by Pawlak and Krawiec using Geometric Semantic GP (Pawlak and Krawiec, 2018). 6-bit, 7-bit, and 8-bit parity circuits were found with RTsSgxm operator after generations using a population of thousand individuals (i.e., evaluations). The average reported circuit size was 298 – 331 gates (3.7x larger than in our case). On average, SOMO requires 5772 evaluations to find a solution for 8-bit parity.

5. Conclusions

We proposed semantically-oriented mutation operator and took a first step towards a more advanced mutation in CGP. The obtained results clearly indicate that the use of the semantically oriented operator is beneficial and significantly improves the search performance of CGP when applied to evolutionary design of combinational circuits.

In our future work, we would like to also use the semantics during mutation of the node functions. This improvement enables us to create a unimodal fitness landscape. In addition, we would like to apply the idea of semantic CGP via semantic mutation to non-Boolean problems.

Acknowledgements.
This work was supported by The Ministry of Education, Youth and Sports of the Czech Republic from the National Programme of Sustainability (NPU II); project IT4Innovations excellence in science-LQ1602.

References

  • (1)
  • Beadle and Johnson (2009) L. Beadle and C. G. Johnson. 2009. Semantically driven mutation in genetic programming. In 2009 IEEE Congress on Evolutionary Computation. 1336–1342. https://doi.org/10.1109/CEC.2009.4983099
  • Castelli et al. (2015) Mauro Castelli, Sara Silva, and Leonardo Vanneschi. 2015. A C++ framework for geometric semantic genetic programming. Genetic Programming and Evolvable Machines 16, 1 (01 Mar 2015), 73–81. https://doi.org/10.1007/s10710-014-9218-0
  • Clegg et al. (2007) Janet Clegg, James Alfred Walker, and Julian Frances Miller. 2007. A New Crossover Technique for Cartesian Genetic Programming. In Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation (London, England) (GECCO ’07). Association for Computing Machinery, New York, NY, USA, 1580–1587. https://doi.org/10.1145/1276958.1277276
  • Coello et al. (1998) Coello Carlos A. Coello, A. D. Christiansen, and A. Hernández Aguirre. 1998. Automated Design of Combinational Logic Circuits by Genetic Algorithms. In Artificial Neural Nets and Genetic Algorithms: Proceedings of the International Conference in Norwich, U.K., 1997. Springer Vienna, Vienna, 333–336. https://doi.org/10.1007/978-3-7091-6492-1_73
  • da Silva and Bernardino (2018) J. E. da Silva and H. S. Bernardino. 2018. Cartesian Genetic Programming with Crossover for Designing Combinational Logic Circuits. In 2018 7th Brazilian Conference on Intelligent Systems (BRACIS). 145–150. https://doi.org/10.1109/BRACIS.2018.00033
  • da Silva et al. (2019) José Eduardo H. da Silva, Lucas A. M. de Souza, and Heder S. Bernardino. 2019. Cartesian Genetic Programming with Guided and Single Active Mutations for Designing Combinational Logic Circuits. In

    Machine Learning, Optimization, and Data Science

    , Giuseppe Nicosia, Panos Pardalos, Renato Umeton, Giovanni Giuffrida, and Vincenzo Sciacca (Eds.). Springer International Publishing, Cham, 396–408.
  • Ffrancon and Schoenauer (2015) Robyn Ffrancon and Marc Schoenauer. 2015. Memetic Semantic Genetic Programming. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (Madrid, Spain) (GECCO ’15). Association for Computing Machinery, New York, NY, USA, 1023–1030. https://doi.org/10.1145/2739480.2754697
  • Goldman and Punch (2013a) B.W. Goldman and W.F. Punch. 2013a. Reducing wasted evaluations in cartesian genetic programming. Lecture Notes in Computer Science 7831 LNCS (2013), 61–72. https://doi.org/10.1007/978-3-642-37207-0_6
  • Goldman and Punch (2013b) Brian W Goldman and William F Punch. 2013b. Length bias and search limitations in Cartesian genetic programming. In Proceedings of the 15th annual conference on Genetic and Evolutionary Computation. 933–940.
  • Goldman and Punch (2015) B. W. Goldman and W. F. Punch. 2015. Analysis of Cartesian Genetic Programming’s Evolutionary Mechanisms. IEEE Transactions on Evolutionary Computation 19, 3 (June 2015), 359–373. https://doi.org/10.1109/TEVC.2014.2324539
  • Hrbacek and Sekanina (2014) Radek Hrbacek and Lukas Sekanina. 2014. Towards Highly Optimized Cartesian Genetic Programming: From Sequential via SIMD and Thread to Massive Parallel Implementation. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (Vancouver, BC, Canada) (GECCO ’14). Association for Computing Machinery, New York, NY, USA, 1015–1022. https://doi.org/10.1145/2576768.2598343
  • Koza (1992) John R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA.
  • Martins et al. (2018) Joao Francisco B. S. Martins, Luiz Otavio V. B. Oliveira, Luis F. Miranda, Felipe Casadei, and Gisele L. Pappa. 2018. Solving the Exponential Growth of Symbolic Regression Trees in Geometric Semantic Genetic Programming. In Proceedings of the Genetic and Evolutionary Computation Conference (Kyoto, Japan) (GECCO ’18). Association for Computing Machinery, New York, NY, USA, 1151–1158. https://doi.org/10.1145/3205455.3205593
  • Miller and Thomson (2000a) J. Miller and P. Thomson. 2000a. Cartesian Genetic Programming. In Proc. of the 3rd European Conference on Genetic Programming EuroGP2000 (LNCS), Vol. 1802. Springer, 121–132.
  • Miller (2019) Julian Francis Miller. 2019. Cartesian genetic programming: its status and future. Genetic Programming and Evolvable Machines (06 Aug 2019). https://doi.org/10.1007/s10710-019-09360-6
  • Miller et al. (2000) Julian F. Miller, Dominic Job, and Vesselin K. Vassilev. 2000. Principles in the Evolutionary Design of Digital Circuits – Part II. Genetic Programming and Evolvable Machines 1, 3 (2000), 259–288.
  • Miller and Thomson (2000b) Julian F. Miller and Peter Thomson. 2000b. Cartesian Genetic Programming. In Genetic Programming. Springer Berlin Heidelberg.
  • Miller et al. (1998) J. F. Miller, P. Thomson, and T. Fogarty. 1998. Designing Electronic Circuits Using Evolutionary Algorithms. Arithmetic Circuits: A Case Study. Wiley, 105–131.
  • Moraglio and Krawiec (2017) Alberto Moraglio and Krzysztof Krawiec. 2017. Geometric Semantic Genetic Programming for Recursive Boolean Programs. In Proceedings of the Genetic and Evolutionary Computation Conference (Berlin, Germany) (GECCO ’17). Association for Computing Machinery, New York, NY, USA, 993–1000. https://doi.org/10.1145/3071178.3071266
  • Moraglio et al. (2012) Alberto Moraglio, Krzysztof Krawiec, and Colin G. Johnson. 2012. Geometric Semantic Genetic Programming. In Parallel Problem Solving from Nature - PPSN XII, Carlos A. Coello Coello, Vincenzo Cutello, Kalyanmoy Deb, Stephanie Forrest, Giuseppe Nicosia, and Mario Pavone (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 21–31.
  • Pawlak and Krawiec (2018) Tomasz P. Pawlak and Krzysztof Krawiec. 2018. Competent Geometric Semantic Genetic Programming for Symbolic Regression and Boolean Function Synthesis. Evolutionary Computation 26, 2 (2018), 177–212. https://doi.org/10.1162/evco_a_00205
  • Sekanina (2017) Lukas Sekanina. 2017. Approximate Computing: An Old Job for Cartesian Genetic Programming? In Inspired by Nature: Essays Presented to Julian F. Miller on the Occasion of his 60th Birthday, Susan Stepney and Andrew Adamatzky (Eds.). Emergence, Complexity and Computation, Vol. 28. Springer, Chapter 9, 195–212. https://doi.org/doi:10.1007/978-3-319-67997-6_9
  • Slany and Sekanina (2007) Karel Slany and Lukas Sekanina. 2007. Fitness Landscape Analysis and Image Filter Evolution Using Functional-Level CGP. In Proc. of European Conf. on Genetic Programming (LNCS), Vol. 4445. Springer-Verlag, 311–320.
  • Vasicek (2015) Zdenek Vasicek. 2015. Cartesian GP in Optimization of Combinational Circuits with Hundreds of Inputs and Thousands of Gates. In EuroGP’15 (Berlin, DE) (LCNS 9025). Springer International Publishing, 139–150.
  • Vasicek (2017) Zdenek Vasicek. 2017. Bridging the Gap Between Evolvable Hardware and Industry Using Cartesian Genetic Programming. In Inspired by Nature: Essays Presented to Julian F. Miller on the Occasion of his 60th Birthday, Susan Stepney and Andrew Adamatzky (Eds.). Emergence, Complexity and Computation, Vol. 28. Springer, Chapter 2, 39–55. https://doi.org/doi:10.1007/978-3-319-67997-6_2
  • Vasicek and Sekanina (2014) Zdenek Vasicek and Lukas Sekanina. 2014. How to Evolve Complex Combinational Circuits From Scratch?. In 2014 IEEE International Conference on Evolvable Systems Proceedings. IEEE, 133–140.
  • Vasicek and Slany (2012) Zdenek Vasicek and Karel Slany. 2012. Efficient Phenotype Evaluation in Cartesian Genetic Programming. In Proc. of the 15th European Conference on Genetic Programming (LNCS 7244). Springer Verlag, 266–278.
  • Vassilev and Miller (2000) Vesselin K. Vassilev and Julian F. Miller. 2000. The Advantages of Landscape Neutrality in Digital Circuit Evolution. In Evolvable Systems: From Biology to Hardware, Julian Miller, Adrian Thompson, Peter Thomson, and Terence C. Fogarty (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 252–263.