1 Introduction
Covering arrays (CAs) are combinatorial objects with applications in software and hardware testing. Recently CAs have been used to detect hardware Trojans Kitsos et al. (2015). A covering array with strength and order is an array over such that every subarray formed by distinct columns contains as a row each tuple over at least once. In testing applications the columns of the CA represent parameters or inputs of the component under test, and a ensures to test all possible combinations of values among any inputs.
Given , , and , the problem of constructing optimal CAs is the problem of determining the minimum for which a exists. This minimum is called the covering array number (CAN) of , , and , and it is denoted by . A similar problem is to find the maximum value of for which a exists; this is denoted by . Values CAN and CAK are related: and .
Finding exact values of has been a very difficult task for general values of , , . Some relevant cases with known values of CAN are these:
Apart from these cases, only a few CANs have been determined. In Colbourn et al. (2010) are listed some optimal CAs for , and in TorresJimenez and IzquierdoMarquez (2016); Kokkala (2017) other CANs were determined by computational search.
There are two main ways to find covering array numbers: by combinatorial analysis and by computational search. In the first case are the works mentioned in the above list of known values of CAN. In the second case we have algorithms that explore the entire search space to determine existence or nonexistence of CAs. These algorithms are limited to small values of , , , since the size of the search space grows exponentially. A first approximation of the size of the search space is , which is the number of matrices over
. Of course, some matrices, like the zero matrix, do not have possibilities of being a CA of strength
and so not all these matrices need to be explored. Although limited to small cases, computational algorithms are one promising option to find CANs for some values of , , and , where there is no adhoc combinatorial analysis.In order to reduce the search space, the computational search uses a nonisomorphic search where only one candidate array is explored for each class of isomorphic arrays. This kind of algorithms are also known as orderly algorithms. There are three isomorphisms in CAs that can be used to bound the search space: row permutations, column permutations, and symbol permutations in a column. Then, any combination of row, column, and symbol permutations produce an equivalent CA, and for purposes of searching for existence only one equivalent CA should be explored. Some algorithms that take advantage of the isomorphisms in CAs are Yan and Zhang (2006); Hnich et al. (2006); BrachoRios et al. (2009); TorresJimenez and IzquierdoMarquez (2016). In these algorithms the CA is constructed element by element, and only the partial arrays that are sorted by rows and by columns are explored; this is done to avoid the exploration of isomorphic arrays obtained by row and column permutations.
The present work addresses the task of finding exact values of , i.e. to find optimal CAs, by means of computational search. The strategy of our searching algorithm is significantly distinct from the strategies of previous algorithms. Instead of attempting to construct the target covering array, say , from scratch or directly in the search space for , , , , our algorithm tries to construct by juxtaposing CAs of strength and columns , , , , where , and by adding to this juxtaposition a column formed by elements equal to for . In this way, our algorithm looks for an array with a certain structure. The algorithm does not look for any array with rows and columns, it looks for an array with rows and columns formed by blocks, where each block is a CA of strength and columns. The fact that each block is a CA of strength and columns greatly reduces the candidate arrays that can be a .
The above searching algorithm is used to determine existence or nonexistence of CAs. From the nonexistence of certain CAs we can derive the optimality of other ones. The main results obtained are the following CANs: , , and . To the best of our knowledge these CANs had not been determined before by any mean. Other computational results are the improvement of the lower bounds of , , , and .
The remainder of the document is organized as follows: Section 2 gives more details about isomorphic and nonisomorphic CAs; Section 3 presents the algorithm to determine the existence of a CA with strength and columns from the juxtaposition of CAs with strength and columns; Section 4 shows an implementation of the crucial step of the algorithm, which is the generation of all possible juxtapositions of nonisomorphic CAs; Section 5 describes the executions of our algorithm to obtain the main results of the work; and Section 6 gives some conclusions.
2 Isomorphic and nonisomorphic CAs
As mentioned before, there are three symmetries in CAs: row permutations, column permutations, and symbol permutations in a column. These operations do not change the coverage properties of a CA, and the CAs obtained by combining these operations are isomorphic to the initial CA. Then, two covering arrays and are isomorphic, which is denoted by , if can be derived from (and vice versa, can be derived from ) by a combination of a row permutation, a column permutation, and a symbol permutation in each column of .
A covering array has isomorphic CAs, because there are possible row permutations, possible column permutations, and possible combinations of symbol permutations in the columns of . Symbol permutations are also called column relabelings or simply relabelings. On the other hand, two CAs and are nonisomorphic if it is not possible to derive from by permutations of rows, columns, and symbols in the columns. The nonisomorphic CAs are the truly distinct CAs. In this work the terms “nonisomorphic” and “distinct” will be used interchangeably when they refer to CAs.
The set of all CAs with the same parameters , , , , can be partitioned in classes of isomorphic CAs. Thus, the relation of being isomorphic is an equivalence relation in the set of all CAs with the same parameters , , , . All CAs in the same class are equivalent, but sometimes it is convenient to take the smallest CA in lexicographic order as the representative of the class.
For given denote by
the vector constructed by arranging in columnmajor order the elements of
. The following definitions taken from TorresJimenez and IzquierdoMarquez (2016) introduce the concept of lexicographic order for CAs, and define CAs with minimum lexicographic order:Definition 1.
Given two CAs and with parameters , , , and , one of the following conditions must occur:

iff for all .

iff exists such that and for all .

iff exists such that and for all
Definition 2.
Given , defines the CA isomorphic to with the minimum lexicographical order iff .
The CA with the minimum lexicographic order in an isomorphism class will be called the minimum of the class. In TorresJimenez and IzquierdoMarquez (2016) it was developed an algorithm called NonIsoCA that generates the minimum of every isomorphism class. This algorithm will be used to generate the nonisomorphic CAs required by our searching algorithm developed in Section 3.
3 Existence of CAs
In this work the existence or nonexistence of a is determined by checking all possible ways of juxtaposing vertically CAs with order , strength , and columns, and by adding to this juxtaposition a column formed by column vectors of constant elements. Determining existence or nonexistence of CAs is the key to find new covering array numbers. The base of the strategy to determine existence of CAs is the following theorem:
Theorem 1.
exists if and only if there exist covering arrays , , , , where , that juxtaposed vertically form a .
Proof.
Assume exists. For let be the number of elements equal to in the last column of . Construct isomorphic to by reordering the rows of in such a way the elements of the last column of are sorted in nondecreasing order. For let be the block of the rows of where the symbol in the last column is . Divide in two blocks: containing the first columns, and containing the last column; then has the following structure:
The juxtaposition of blocks form a because has strength ; then, to complete the first part of the proof we need to show that blocks are CAs of strength . Index columns of starting from 0, so the last column of has index . Any combination of columns containing the last column of covers in block all tuples of the form over . Thus, every combination of columns from the first columns of covers all tuples over in every block , and therefore .
Now, suppose there are covering arrays , , , , whose vertical juxtaposition forms of strength , where . Let be the column formed by concatenating vertically elements equal to for . Because every is a CA of strength we have that for any submatrix formed by columns of and by covers all tuples of the form . Then, any sumbatrix formed by columns of and by column covers all tuples over , and therefore the horizontal concatenation of and is a . ∎
By Theorem 1 if a exists then it can be constructed by juxtaposing vertically CAs with strength and columns. Also, if does not exists then there are no CAs with strength and columns that juxtaposed vertically form a .
The algorithm developed in this work to determine the existence of verifies all possible juxtapositions of CAs with strength and columns to see if one of them produces a . In the negative case, does not exist; and in the positive case exists and the algorithm generates all nonisomorphic .
The first step of the algorithm is to determine the multisets =, , , of elements such that and . These multisets will be called valid multisets. To determine for example if exists we need to check all juxtapositions of three CAs with strength two, four columns, and number of rows given by . In this case is the unique valid multiset because , and therefore if exists is necessarily composed by three . On the other hand, to determine if exists, the multisets to consider are and , because can be composed by two CAs of nine rows and one CA of eleven rows, or by one CA of nine rows and two CAs of ten rows.
The second step of the algorithm is to generate the nonisomorphic CAs (i.e., the minimum CAs of every isomorphism class) with strength , columns, and number of rows given by a valid multiset . From each nonisomorphic CA the other members of its isomorphism class will be derived by permutations of rows, columns, and symbols. To construct the nonisomorphic CAs we can use the NonIsoCA algorithm of TorresJimenez and IzquierdoMarquez (2016), or any other algorithm for the same purpose.
Given a valid multiset , let () be the set of all nonisomorphic . From the CAs in the sets will be generated all juxtapositions of CAs whose number of rows are given by . In the example with , the sets and contain the nonisomorphic , and the set contains the nonisomorphic . Now, let be the Cartesian product of the sets ; then contains all possible ways to combine the nonisomorphic CAs with number of rows given by .
The next step of the algorithm is to check all juxtapositions derived from a tuple of . For a tuple , let denote the juxtaposition of the CAs in . From this array it will be generated all arrays where each is derived from exactly one by permutation of rows, columns, and symbols in the columns; in other words, indices are a permutation of , and is isomorphic to .
The total number of arrays derived from one tuple is . Each array is checked to see if it is a . If this is the case, then exists by Theorem 1, and this CA is obtained by adding to the column .
For each tuple of all possible arrays are generated; then, all possible juxtapositions of CAs with strength , columns, and number of rows given by a valid multiset are explored. Since this is done for every valid multiset , we have that all possible juxtapositions of CAs with strength and columns are explored.
The number of juxtapositions verified to determine the existence of may be very large, however some juxtapositions produce isomorphic arrays, and to accelerate the search we need to skip as many isomorphic arrays as possible. Fortunately, the number of arrays created for a tuple of can be reduced considerably. Consider the horizontal of an array and the column , denoted as :
We can reorder the rows of in such a way the array derived from is placed in the first rows of , the array derived from is placed next, and so on. This permutation of rows produces an array isomorphic to , and by a permutation of symbols in the last column of it is possible to transform the last column of in :
Therefore, the arrays to be generated from a tuple = , , , of are those arrays where for the array is derived from by permutations of rows, columns, and symbols.
Another reduction in the number of arrays is possible: we can permute the first rows of , and permute the columns and symbols in the entire array to get an array , where and are another CAs isomorphic to the original arrays . Thus, the following arrays are isomorphic:
In this way, the arrays to be generated from a tuple , , , of are those arrays where is fixed, and for the array is derived from by permutations of rows, columns, and symbols. However, note that block of is complemented with a column vector formed by elements equal to . Then, any row permutation of produces an array isomorphic to . On the contrary, column and symbol permutations in do not produce in general arrays isomorphic to . Therefore, the only arrays that are necessary to explore are those derived from by permutations of columns and symbols. In this way, we have reduced the number of arrays to be generated from a tuple from to .
Algorithm 1 implements the algorithm to determine existence of CAs. From the input parameters and are obtained the number of columns and the strength of the CAs to be juxtaposed. The generation of the valid multisets can be accomplished without difficulty, but it requires to known the value of . The construction of the sets requires the computation of the nonisomorphic , which as mentioned before can be done with any algorithm to generate distinct CAs. The key function is generate_juxtapositions(), where arrays are generated from a tuple of the set ; this function will be described in Section 4.
For each constructed by generate_juxtapositions() we obtain the minimum array of the isomorphism class to which belongs, and then is added to the set of distinct CAs. To obtain we assume the existence of a function minimum which computes and returns for given . This function can be derived from a slight modification of the function is_minimum of TorresJimenez and IzquierdoMarquez (2016).
4 Generation of juxtapositions
The crucial step of the algorithm to determine the existence of is to generate all arrays from a given tuple of CAs , , , . Recall that in each , the array is fixed, and for the array is derived from by permutations of columns and symbols. After generating an array , the algorithm checks if is a .
This section presents an algorithm to perform this crucial step. The algorithm constructs one column at a time, validating that each new column forms a CA of strength with the columns previously added to . This is done to avoid the exploration of arrays with no possibilities of being a . The algorithm starts by constructing the following array , in which block is fixed, and blocks are unassigned or free; later on these arrays will be filled with arrays derived from by permutations of columns and symbols:
For let be the columns of , and let , , , be the columns of . Then, the previous array is equivalent to this one:
The algorithm fills column 0 in all free blocks, then it fills column 1 in all free blocks, and so on. In this way, columns are filled first, then columns are filled, and so on. When the first columns of all free blocks have been filled or assigned, the algorithm checks if they form a CA of strength . Columns are indexed from 0, so the first columns of are formed by columns , and by columns for .
If the first columns of form a CA of strength , then the algorithm advances to the next column of the free blocks, and column is assigned, then column is assigned, and so on until column is assigned. After that, the algorithm verifies if the current first columns of form a CA of strength . In the negative case the current value of is replaced by its next available value to see if now the first columns of form a CA of strength . This is done for all available values of , and when all values are checked the algorithm backtracks to and assigns to it its next available value; after that, the algorithm advances to to check again all its available values.
To construct all possible arrays the algorithm fills the free block with all isomorphic CAs derived from by permutations of columns and symbols. Thus, the possible values for a column of are the columns obtained by permuting symbols in the columns of ; so the number of available values for a column of is . When the first columns of have been assigned the number of available values for is , which are the relabelings of the columns of not currently assigned to one of the first columns of .
In every free block the algorithm works as follows: columns of are added to in such a way gets all columns in order; then for a fixed value of , column gets in order all columns of distinct to the one assigned to ; and for fixed values of and , column gets in order all columns of not currently assigned to or ; the same applies for the other columns of . In this way gets all CAs derived from by permutation of columns.
However, for each permutation of columns of the algorithm of Section 3 requires to test all possible symbol permutations in the columns of . Symbol permutations are integrated in the following way: suppose the first columns of have been assigned, and suppose the next free column of gets assigned column of ; we can consider that the current value of is the identity relabeling of ; the next values to assign to are the other relabelings of . When all relabelings of are assigned to , the next value for is the identity relabeling of the next column of that has not been assigned to .
The function generate_juxtapositions() of Algorithm 2 receives as parameter a tuple of CAs. The work of this function is to initialize the fixed block of , and to initialize with FALSE the elements of a matrix called assigned; this matrix is used to record which columns of are currently assigned to a column of . The last sentence of the function generate_juxtapositions() is a call to the function add_column(), which fills the free blocks with CAs derived from by permutations of columns and symbols. The function add_column() is called from generate_juxtapositions() with arguments 1 and 0, because the first column to fill in array is column of the free block , or .
The function add_column() of Algorithm 3 receives as parameters an index of a free block and an index of a column of the free block; the work of the function is to set column . The variable is used as an alias of the block of where a CA derived from will be placed. The function relies on recursion to assign column 0 of every free block, then to assign column 1 of every free block, and so on. In addition, in every free block recursion allows to test in column all columns of not currently assigned to a column of ; the main for loop iterates over all columns of , but the body of the loop is executed only for those columns for which assigned is equal to FALSE. Recursion also allows to check in order the symbol permutations of the column of assigned to column of . If a is constructed, then column is appended to it to form ; finally, the function obtains and adds it to the set of nonisomorphic .
For a tuple of CAs , Algorithm 2 and its helper function Algorithm 3 generate in the worst case arrays , since array is fixed and are derived respectively from by permutations of columns and symbols. However, the condition that every new column added to the partial array must form a CA of strength with the previous columns of reduces the number of arrays explored. For example if the condition fails at the column with index , then in each free block we skip the remaining permutations of columns for the free columns , plus the associated column relabelings.
We can see in Algorithm 3 what makes our algorithm significantly distinct from previous ones. The target covering array is not constructed element by element, but subcolumn by subcolumn, where a subcolumn is a column of a CA of strength . Nevertheless, our algorithm requires the construction of the nonisomorphic CAs of strength and columns, which could had been constructed element by element. However, the cost of constructing the nonisomorphic plus the cost of exploring the juxtapositions of CAs derived from then by permutations of columns and symbols, is smaller than the cost of constructing element by element, if the number of distinct is not very large, as we will see in the next section.
5 Computational results
The relevant computational results obtained in this work are the covering array numbers , , and ; as well as the uniqueness of ; and the improvement of the lower bounds of , , , and . All these results are consequences of the nonexistence of certain CAs.
5.1
The current lower bound of is 30 Colbourn et al. (2010), and its current upper bound is 32 TorresJimenez and RodriguezTello (2012). In this section we prove that no and no exist, and therefore .
By Theorem 1 if exists, then there exist two covering arrays and such that , and their vertical juxtaposition forms a CA of strength . Now, the only possibility for the values of and is because Colbourn et al. (2010); Nurmela (2004); so the unique valid multiset in this case is . The NonIsoCA algorithm gives two distinct , and using these CAs Algorithm 1 did not find a .
Similarly, to determine the existence of the unique valid multiset is . Algorithm 1 tested the juxtapositions of the two nonisomorphic with the 44,291 nonisomorphic reported by the NonIsoCA algorithm. Also in this case no was found. Therefore, is optimum, and .
Conceptually it is possible to run the NonIsoCA algorithm to prove the nonexistence of and directly in strength four. However, it is more convenient to use the NonIsoCA algorithm to construct the nonisomorphic CAs with strength and columns required by Algorithm 1 to search for and . In a machine with processor AMD Opteron™ 6274 at 2.2 GHz the NonIsoCA algorithm takes approximately 1.38 hours to construct the two distinct , and takes about 937 hours to construct the 44,291 distinct . However, the execution time of Algorithm 1 on the same machine is only 3 seconds for , and 16 hours for . Thus, the total time to determine that was approximately 955 hours. In contrast, we attempted to construct the nonisomorphic
using the NonIsoCA algorithm, but we aborted the search after 3 months of execution, because based on the partial results we estimated that the execution would not end any time soon.
In the process of proving the optimality of almost all execution time was consumed in constructing the 44,291 nonisomorphic . We could use Algorithm 1 to construct these CAs; however in this case Algorithm 1 is not the best option because there are too many CAs with strength and columns to be combined to form a . Since , we can construct a by juxtaposing a and a , or by juxtaposing two . The number of nonisomorphic is only 26, but there are 377,177 nonisomorphic , and distinct . Thus, in Algorithm 1 the function generate_juxapositions() would be called times.
The new covering array number has important consequences. In TorresJimenez et al. (2015b) it was reported a Tower of Covering Arrays (TCA) beginning with
Comments
There are no comments yet.