Abstract
We study the space that consists of all nonsingular binary matrices, that is, . The space is quite important in that it is used for the change of basis in binary representation, which is the encoding typically adopted in genetic algorithms. We analyze the properties of and theoretically design possible encodings and their corresponding recombination operators for evolutionary algorithms. We present approaches based on elementary matrices of linear algebra as well as typical two-dimensional ones.
1. Introduction
An matrix is called binary if [1]. Binary matrices have been widely used to deal with the adjacency of a graph [2–5]. In particular, Anderson and Feil [2] transformed the light bulb puzzle into the problem of solving a linear system from its graph structure, where and . Then the solution could be obtained by computing the inverse of , that is, . Binary matrices are also useful in dealing with the cut/cycle subspace of a graph, which is a vector space over [3, 6]. They can also be used to represent a change of basis of a vector space over in combinatorial problems [7, 8]. A change of basis plays an important role in the performance of evolutionary algorithms [8–12]. It can be represented as a nonsingular matrix, and hence we need to consider the space composed of all nonsingular binary matrices. In this paper, we investigate the space of nonsingular binary matrices, . Table 1 shows the sizes of spaces according to small . The size of is . Those of in Table 1 are obtained by an exhaustive search over . Each matrix in is verified whether it is singular or nonsingular by Gauss-Jordan elimination. grows exponentially as matrix size increases.
In this paper, we focus on the mathematical designs of encodings and their corresponding recombination operators in for evolutionary algorithms. The remainder of this paper is organized as follows. In Section 2, typical 2D encoding and recombination operators, in which we can intuitively and easily devise, are introduced. In advance of the typical 2D approaches, ones using elementary matrices of linear algebra are presented in Section 3. In Section 3.1, we introduce the elementary matrix of as a preliminary. If we represent a nonsingular binary matrix as a product of elementary matrices, the feasibility of offspring is naturally preserved. As representation based on the product of elementary matrices, we propose the representation by a variable-length linear string in Section 3.2 and that by directed acyclic graph in Section 3.3. When we consider a nonsingular matrix as a product of elementary matrices, the representation is not unique. Moreover, it is not so easy to know how many equivalent representations exist for a nonsingular binary matrix. In Section 3.4, nontrivial equivalences are investigated to help analyze the same genotypes for a phenotype. Finally, we give our conclusions in Section 4.
2. Approaches Based on 2D Encoding
The most intuitive and natural representation for a nonsingular binary matrix is 2D binary encoding. Or more simply, we can use just length- binary linear string. When using linear encoding, we can adopt traditional genetic operators such as -point crossover and uniform crossover for recombination operators.
However, a 2D encoding/crossover pair can reflect more geographical linkages of genes than 1D encoding/crossover pairs [13]. Cohoon and Paris [14] proposed a 2D crossover which chooses a small rectangle from one parent and copies the genes in the rectangle into the offspring, with the rest of the genes copied from the other parent. Anderson et al. [15] suggested the block-uniform crossover on grid. It divides the grid into blocks at random; each block of one parent is interchanged randomly with the corresponding block of the other parent based on a preassigned percentage.
Although 2D encoding can preserve more geographical relationships among the genes, when traditional straight-line-based cutting strategies are used, the power of new-schema creation is far below that of the crossovers on linear encodings [16].
Geographic crossover was suggested to resolve this problem [13, 16, 17]. In the case of a 2D encoding, it chooses a number of monotonic lines, divides the chromosomal domain into two equivalence classes, and alternately copies the genes from the two parent chromosomes. Figure 1 shows an example using geographic crossover in 2D encoding of binary matrices. By combining 2D representation and geographic crossover, we can expect both reduced information loss in the stage of encoding and the power of new-schema creation.
| (a) Parent 1 () |
| (b) Parent 2 () |
| (c) An offspring generated by geographic crossover from and (not invertible, i.e., infeasible) |
| (d) Repaired offspring of (c) (invertible, i.e., feasible) |
However, an inevitable defect of this typical 2D approach is that it needs feasibility check and repair mechanism. We can see an example case from Figure 1(c). In this case, the offspring is not invertible. One of the possible repair mechanisms is finding maximal linearly independent rows (or columns)—this can be conducted by Gauss-Jordan elimination—and replacing linearly dependent rows/columns by other available basis elements (see Figure 1(d)). Algorithm 1 shows the repair mechanism. But it is too time consuming since Gauss-Jordan elimination takes time.
| ||||||||||||||||||||||||||||||||||||||||||||
3. Approaches Based on Elementary Matrices
In this section, we propose novel methods inspired by the fact that all the invertible matrices can be represented as products of elementary matrices [18].
3.1. Elementary Matrix in
Before introducing the elementary matrix of , we define elementary row operation.
Definition 1. Let . Any one of the following two operations on the rows of is called an elementary row operation: (i)interchanging any two rows of and(ii)adding a row of to another row.
Elementary row operations are of type 1 or type 2 depending on whether they are obtained by (i) or (ii) of Definition 1. An elementary matrix over field is a matrix obtained by performing an elementary row operation on the identity matrix . The elementary matrix is said to be of type 1 or type 2 according to whether the elementary row operation performed on is of type 1 or type 2, respectively. We can easily check the following property.
Proposition 2. For each elementary matrix , , where is the identity matrix.
Proof. It is trivial for the cases of type 1. Since and in , for any row vector . Hence, the identity also holds in the cases of type 2.
We define as the elementary matrix of type 1 that interchanges the th row and the th one (). We also define as the elementary matrix of type 2 that adds the th row to the th one (). For example, consider cases; that is, . There are three elementary matrices of type 1, , , and : And there are six elementary matrices of type 2, , , , , , and :
Proposition 3. The number of elementary matrices in is . In detail, let and be the sets of all elementary matrices of type 1 and of type 2 in , respectively. Then, and .
Proof. Since for each pair, . Since for each pair, .
It is known that every invertible matrix is represented as a product of elementary matrices [18]. Hence, if we represent a solution in as a product of elementary matrices, we get a big advantage of preserving feasibility.
3.2. Variable-Length Linear String
In the case of using elementary matrices, we can consider the representation by variable-length linear string, of which each element is an elementary matrix. Matrix in Figure 1(a) is when represented as a product of elementary matrices. Similarly, matrix in Figure 1(b) becomes . So their new encodings are given in Figure 2(a).
As recombination operators for this encoding, any recombinations for variable-length string can be used. Among such recombination operators, geometric crossover [19–24] by sequence alignment [25–27] is expected to be effectively performed. The string can be regarded as a sequence having the fixed number of elementary matrices. Geometric crossover by sequence alignment will be helpful for offspring to preserve features inherent in parents.
As a distance for variable-length string, there is edit distance : the minimum number of insertion, deletion, and replacement of single elementary matrix to transform one string into the other. The geometric crossover associated with this distance is called the homologous geometric crossover [25]: two strings are aligned optimally before recombination. Alignment here means allowing parent strings to be stretched to match better with each other. Formally stretching strings means interleaving “—” anywhere and in any number in the strings to create two stretched strings of the same length that have minimal Hamming distance. In our example, when we recombine and , we need to align them optimally first: and (see Figure 2(b)). Notice that the Hamming distance between the aligned strings is less than that between the nonaligned ones.
After the optimal alignment, one does the normal crossover and produces a new stretched string. The offspring is obtained by removing “—”, so by unstretching the string (see Figure 2). In the example of Figure 2, we can easily check .
Now we assume that insertion, deletion, and replacement have weights of 1, 1, and 2 in edit distance , respectively. Then, when , and . In general, the following proposition holds.
Proposition 4. Let , , be in . Then ; that is, is translation-invariant.
Proof. Let be an elementary matrix in . By the definition of edit distance , . Here the equality holds when the last elementary matrices of and are aligned. Analogously, we can obtain the inequality of . By Proposition 2, , and then . Hence, . Since nonsingular binary matrix can be represented by a product of elementary matrices, by induction, . Similarly, we can show that .
3.3. Directed Acyclic Graph
When the representation by elementary matrices is considered as a sequence of elementary matrices, the representation for a matrix may not be unique. For example, and . Moreover, it is not easy to know how many equivalent representations exist for a nonsingular matrix. However, we could find an obvious equivalence as follows.
Proposition 5. Let be . Let , , and . Then, .
Proof. Let be an binary matrix in which is the th row vector. It is enough to show that = . changes only two row vectors and of , and changes only two row vectors and of . Since by assumption, the th and th row vectors of are the same as and , respectively, and the th and th row vectors of are the same as and , respectively. Also, does not change the th and th row vectors of . Analogously, does not change the th and th row vectors of . The th and th row vectors of and are the same as those of , respectively, and the th and th row vectors of and are the same as those of , respectively. Hence, = .
Figure 3 shows the noninterchangeability among the sets s. Each set has only one interchangeable set among the total 6 sets. It seems that such interchangeable cases are rare. However, it is not true for large . For arbitrary chosen elementary matrices and , the probability of is as follows: Figure 4 shows the probabilities of according to matrix size . We can see that is close to 1 for sufficiently large . It means that the cases of have very frequently appeared for large .
Based on the equivalence of Proposition 5, we can represent an invertible matrix by using a directed acyclic graph (DAG) with elementary matrices as its vertices. Each edge means noninterchangeability between two elementary matrices. Consider . We assume that the DAG structure of is as follows:
Then, is a topological order of the above DAG. The following is also another topological order of the above DAG
When we use this DAG representation for nonsingular binary matrices, we can apply known DAG recombination such as [28] or design geometric operators based on the similarity of DAGs [29].
Figure 5 shows the DAG representation of parents given in Figure 2 and an offspring generated by a simple DAG recombination. We can see that DAG representation reflects the similarity of parents more than that based on variable-length string.
| (b) An offspring generated by a simple DAG recombination |
| (c) Matrix form of the offspring |
3.4. Nontrivial Equivalences
We could also find more sophisticated equivalences. One class of them is about the exchange of two elementary matrices.
Proposition 6 (exchange rules). For each , , such that , , and , the following five exchange rules hold. (i), (ii), (iii), (iv), and (v).
Proof. Let be an binary matrix in which is the th row vector; that is, .(i)It is enough to show that the th row vector of is the same as that of . First, consider the left side: . . Now, consider the right side: . .(ii)It is enough to show that the th and th row vectors of are the same as those of . Without loss of generality, we assume that . First, consider the left side: . . Now, consider the right side: . .(iii)It is enough to show that the th, th, and th row vectors of are the same as those of . Without loss of generality, we assume that . First, consider the left side: . . Now, consider the right side: . .(iv)It is enough to show that the th, th, and th row vectors of are the same as those of . Without loss of generality, we assume that . First, consider the left side: . . Now, consider the right side: . .(v)It is enough to check the th, th, and th row vectors of , , and . We can easily check that the th, th, and th row vectors of are , , and , respectively. Similarly, the th, th, and th row vectors of and are the same as those of .
Another class is about the compaction of three elementary matrices into two ones.
Proposition 7 (compaction rules). For each , , such that , , and , the following two compaction rules hold. (i) and (ii).
Proof. Let be an binary matrix in which is the th row vector; that is, .(i)It is enough to show that the th, th, and th row vectors of are the same as those of . Without loss of generality, we assume that . First, consider the left side: . . Since , . Now, consider the right side: . .(ii)It is enough to show that the th, th, and th row vectors of are the same as those of . First, consider the left side: . . . Now, consider the right side: . .
The encodings of matrices and given in Figure 2 can be reduced to other forms by using the presented propositions as follows:
From the reduction, the edit distance between both parents decreases by 2. This reduction process can be understood as a kind of normalization [23]. Figure 6 shows a recombination example with the reduced encodings of parents. It shows that reduced encodings can produce more similar offspring to parents.
4. Conclusions
We presented several ideas to deal with the space of nonsingular binary matrices, , in evolutionary algorithms. This space is important in that it is used for the change of basis of standard binary encoding. As typical approaches, we represented a matrix in just as an array of matrix elements. Two-dimensional binary encoding or length- binary linear string can be used in these approaches. However, these approaches need the feasibility check of offspring together with their repair mechanism. We illustrated this nature by an example.
We presented some novel methods using elementary matrices, which have an advantage of preserving feasibility. An invertible matrix can be represented as a product of elementary matrices. Variable-length linear string and DAG (directed acyclic graph) representation are proposed to represent each nonsingular binary matrix that could be understood as a product of elementary matrices. Different products of elementary matrices often represent the same matrix, so we tried to get more information about the relation between representations with the same phenotype by finding some nontrivial equivalences.
Future work will be directed to apply these results to the problem of finding good bases of binary encoding in evolutionary algorithms.
Disclosure
A preliminary version of this paper appeared in the Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation, pp. 855–858, 2009.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.