Abstract
This study proposes a neural network-family competition genetic algorithm (NN-FCGA) for solving the electromagnetic (EM) optimization and other general-purpose optimization problems. The NN-FCGA is a hybrid evolutionary-based algorithm, combining the good approximation performance of neural network (NN) and the robust and effective optimum search ability of the family competition genetic algorithms (FCGA) to accelerate the optimization process. In this study, the NN-FCGA is used to extract a set of optimal design parameters for two representative design examples: the multiple section low-pass filter and the polygonal electromagnetic absorber. Our results demonstrate that the optimal electromagnetic properties given by the NN-FCGA are comparable to those of the FCGA, but reducing a large amount of computation time and a well-trained NN model that can serve as a nonlinear approximator was developed during the optimization process of the NN-FCGA.
1. Introduction
Nowadays, with the improving performance of high-speed computing systems and progress in computational electromagnetics, the computationally intensive full-wave EM analysis (i.e. method of moments (MoM) [1], finite-difference time-domain (FDTD) method [2, 3], and finite-element method (FEM) [4–6]) has become practical and thus has been performed widely in microwave circuits and electromagnetic-based designs. To obtain the optimal design variables for microwave (i.e., antennas [1, 7–9] and filters [10, 11]) and optoelectronic (i.e., optical waveguides [12] and optical fibers [13]) devices and components, the use of numerical simulation and full-wave analysis to obtain the optimal design variables has become indispensable. However, for most electromagnetic problems, propagation effects such as strong mutual coupling of EM waves are unavoidable; therefore, a simultaneous optimization of the various design variables is required. Moreover, most complex electromagnetic problems have highly nonlinear objective functions involving a large search space and thus these functions are difficult to be evaluated. Consequently, it is nearly impossible to optimize all design variables simultaneously and obtain results in the global optimum through the traditional gradient-based local-search optimization methods, such as the hill-climbing method or the Simplex method. As a result, there has been an increasing interest in the use of evolutionary computing as the optimization strategy over the past several years [1, 6–18].
Evolutionary computing is a branch of artificial intelligence (AI) and is based on direct and adaptive search techniques. The algorithms used in these evolutionary computing strategies are, in general, inspired by biological or sociological motivations. A significant advantage for using these strategies is that most evolutionary computing strategies are conceptually very straightforward and do not require any information pertaining to the gradient, which in turn allows the global optimality of objective functions that are rough, discontinuous, and multimodal. In particular, the genetic algorithm (GA) has been rapidly accepted in the electromagnetic research community and has been extensively studied in a variety of electromagnetic optimization problems, such as the optimization of broadband or multiband antennas [1, 7–9], functional filters [10], high efficiency absorbers [6, 14], and photonic crystals [15–17].
The GA is originally proposed by Holland and his colleagues in 1975. It borrows the natural evolution mechanisms to find out the optimal solutions to the user-defined problems [19]. It starts with a randomly generated initial population covering the entire range (also known as the search space) and then undergoes the reproduction processes that combine the biologically-analogical operators of selection, cross-over, and mutation to generate a new and fitter population of potential solutions. The reproduction process is iterated by simulating the paradigms of natural selection and survival of the fittest until the fittest solution is obtained. So far, a great deal of effort has already been expended to precipitate the maturity of both the GA and the computational electromagnetic, as well as combining these two different techniques to perform rapid optimization of practical electromagnetic designs. Although the GA has been increasingly used in providing optimal or nearly optimal solutions to a number of optimization problems, the algorithm still has a certain probability of being trapped in a local optimum [20, 21], albeit this probability is smaller than that in the traditional optimization methods. Therefore, a number of studies aiming to improve the algorithm’s search ability have been done on the adaptive search rules and the fundamental theory of the GA. Very recently, the family competition principle has been introduced [18, 20, 21] to the GA with the intent of improving the quality of the solution and increasing the probability of locating the global optimum. In the FCGA, the family competition can be seen as the local competition of each specific area of the search space (local search) and the selection in the reproduction process can be seen as the global competition in the universal tournament. Although the previous works have shown that the FCGA can yield significant improvement in the optimization results, it however requires many additional simulations in the time-consuming family competition reproduction process.
In this paper, we intend to propose a fast and effective optimization strategy-neural network-family competition genetic algorithm (NN-FCGA) to solve both the computing time and search result accuracy problems. The motivation behind the NN-FCGA is that it combines the good approximation performance of neural networks (NN) for the complicated analysis and the robust and effective optimum search ability of the FCGA to achieve the rapid and effective electromagnetic optimization.
This paper is organized in the following manner. The next section will introduce the GA, FCGA, NN, and NN-FCGA. In Section 3, the GA, FCGA, and NN-FCGA are applied to design a low-pass planar filter and a polygonal electromagnetic absorber. Finally, the conclusion addresses the advantage of the NN-FCGA compared to the traditional GA and FCGA. Although, in this study, the NN-FCGA is used for electromagnetic optimization, it can be readily extended to other physical applications such as microelectronics [22], material science [23], mechanical and fluid dynamics [24, 25], and general-purpose optimization problems.
2. Neural Network-Family Competition Genetic Algorithm
2.1. Family Competition Genetic Algorithm
As mentioned earlier, the GA was inspired by evolutionary genetics. In the traditional real-coded GA, each variable that requires optimization is regarded as a gene, which is represented by a floating-point number. All variables are stored in a vector known as a chromosome??(or an individual), which is the basic entity in the GA’s evolution process. Each individual fully describes a possible solution. At the start of the algorithm, a certain number of individuals are randomly generated to form the initial population. In each generation, the fitness of each individual in the current population is evaluated and ranked to determine which one is more likely to survive and better suited in the subsequent generation. For most of the electromagnetic optimization problems, the fitness deals with the scattering response or the field and energy distributions. To direct the evolution toward the optimal solution with respect to the predefined fitness function, the next generation is produced by the reproduction process that utilizes the biologically analogical operators of selection, crossover, and mutation. These operators make the GA a parallel iterative algorithm with certain learning abilities. The flowchart for the GA is illustrated in Figure 1(a). In selection, pairs of individuals from the current generation are selected as parents for mating. Individuals with higher fitness values are more likely to be selected for the creation of the next generation. In crossover, parents interchange their genetic material with a crossover probability to create a new pair of individuals called children for the use in the next generation. As in nature, mutation is also necessary to maintain the diversity in the population and explore the solutions that are not yet present. In mutation, each gene is possibly replaced by an arbitrary number with a mutation probability . The three operators—selection, crossover, and mutation—are repeatedly applied to the current generation until the population size of the new generation is the same as that of the current generation. These GA processes are iterated either until the design goal is reached or until no further improvements can be found in the fitness values of the best individual. For further details, see Holland [19] and Rahmat-Samii [26].
(a)
(b)
Recently, the concept of the family competition has been added to the reproduction process of evolutionary algorithms [18, 20, 21]. The FCGA’s algorithm is mainly based on the GA where the population size of each generation remains the same, while the family competition is conducted in the reproduction process. In the family competition process, a father in a family will mate with several individuals (mothers) that are randomly selected from the current population to create several children (more than six children were used in the previous experiments [18, 20, 21]), resulting in a family that includes the family’s father and his children. Once families are formed, all the children in each family will undergo mutation and eventually the two fittest members will be the only survivors. The reproduction process for the FCGA is illustrated in Figure 1(b). In such a manner, the FCGA precludes the premature convergence and prevents the search space from being bound in the local optimum. Despite these advantages, however, the most serious drawback of the FCGA is the increasing computation time relative to the GA due to the fitness evaluation required for all members in the families. This is especially critical for those cases where the complicated analysis or time consuming simulation, such as the full-wave EM analysis, is required for the fitness evaluation.
The settings used in the GA-based optimization algorithms (including the GA, FCGA, and NN-FCGA), unless otherwise noted, are detailed below and will be used throughout this study. In this study, the population size and total number of generations were set to 40 and 60, respectively. For the selection procedure, the tournament-selection strategy is used, where four individuals are randomly chosen and the best two individuals are selected as the parents for mating. For the crossover procedure, the intermediate-crossover??strategy is used, where a child individual is created by a weighted average of the parent: where is a uniform random number from interval .
For the mutation procedure, an adaptive Gaussian mutation is used, where each gene is possibly replaced by an arbitrary number in the vicinity of the original individual with the mutation probability set to : where is the step size: where indicates the gene in the individuals, and represent the upper and lower bound of each gene, respectively, and is the population size in the current generation. In addition, the elitism strategy is used, where the fittest individual from the current generation survives and is directly copied into the new generation. For the FCGA and NN-FCGA, each father is allowed to create ten children; ultimately, two fittest family members survive.
2.2. Neural Network-Family Competition Genetic Algorithm
Neural network (NN), which is also a branch of AI, is a computational model based on the concept of biological neural networks, as shown in Figure 2. It consists of an interconnected group of computing units called neurons and processes information using a connectionist approach. It is a powerful tool for modeling the nonlinear or complex relationships between the inputs and the outputs and has been used extensively in various fields [27–30]. Due to the good approximation ability of NN, we intend to utilize NN to provide the rapid fitness evaluation, thus reducing the computation cost, resulting from the family competition process in the FCGA.
The steps in the algorithm used in the NN-FCGA hybrid method for electromagnetic optimization are described as follows. (1)In the initial population, many individuals are randomly generated and their fitness values are evaluated by performing the full-wave EM analysis.(2)All individuals’ gene information and their corresponding fitness values are collected as the NN’s training samples to construct an NN approximation model. The values of all genes in an individual and their corresponding fitness values can be seen as the input and output of NN, respectively.(3)In the next generation, the reproduction operation in the FCGA is performed to form a family. The fitness value of each family member is approximated by the NN instead of the relatively time-consuming full-wave EM analysis. The two fittest members are selected for use in creating a new generation.(4)To improve the accuracy of NN, the fitness values of the elites (the champions in each of the families) are recalculated using the full-wave EM analysis. The updated fitness and gene information of the elites are then feedback to the NN in order to yield a more accurate approximation model.
Steps (2) to (4) are repeated until the stopping criterion of NN-FCGA is satisfied; in our case, the criterion is met when 60 generations have been generated.
In the generation of NN-FCGA, all genes and their associated fitness values evaluated by the full-wave analysis from the initial to generation are reserved as the training samples for the NN. As a result, the NN’s approximation results become more and more accurate with increasing the number of generations; this is because more training samples are available. At the end of the reproduction process, the best individual and an accurate NN approximation model are obtained.
Our study utilizes a typical back-propagation neural network (BPNN) composed of three basic layers: input, hidden, and output. The diagram of the BPNN is shown in Figure 2. The network input represents the genes of an individual and the output represents their corresponding fitness values. The presented model is developed using the error back-propagation algorithm. Using this algorithm, input data is first passed through the network by using a set of weights, which can be seen as the connection strength. The output of each neuron is the weighted sum of its inputs filtered by a sigmoid function. Here, we use the indices , , and k for the neurons corresponding to the input, hidden, and output layers, respectively, and for the input and output of the neuron, respectively. The input-output relationships of the neurons are given as follows: where represents the weight from the () neuron of the input (hidden) layer to the () neuron of the hidden (output) layer and represents the threshold parameter for each neuron. Initially, the weights are randomly given. Then, the final outputs and training data are compared and the training error function is calculated: where is the number of training samples, and represent the desired output and computed output for the inputting sample, respectively. The renewed weights are then updated by where in and in are the learning rate and momentum factor, respectively. The setting of NN, unless otherwise noted, is listed in the following and will be used throughout this study. Here, a learning rate of momentum factor of , and thresholds of are used. The number of neurons for the input and output layers is the number of genes in an individual and 1 , respectively, whereas that of the hidden layer is determined by the problems ( in this case). The NN is repeatedly trained until the convergence criterion is satisfied (5000 iterations in this case) and then it is used for fitness evaluation in the NN-FCGA.
The various algorithms (GA, FCGA, NN, and FCGA) were written in the open-source program, Python; a computer intelligence technique optimization platform, CITO lab written in Python was developed at the Skywatcher Lab for the intercommunication between the CITO lab (optimizer) and the EM solver (simulator). In each generation, the individuals appear in the optimization processes and the associated fitness values are automatically saved to the database in the CITO lab. Hence, the fitness values of the same individuals that reappear in the subsequent generations will not be recalculated, thus reducing computation time.
3. Results and Discussions
In this section, a low-pass filter and three polygonal electromagnetic absorbers as design examples are presented for the NN-FCGA. To provide a baseline for comparison, these examples are also presented for the FCGA and GA.
3.1. Planar Band-pass Filter Design
Filtering components have been widely used in microwave circuits. In the following, we will study the three GA-based optimization algorithms by applying them to a planar filter. A low-pass planar filter model [31] based on the concept of multiple section folding is illustrated in Figure 3. Such a filter is created using microstrip lines and is mainly composed of three metallic patches in the middle section and a impedance-matched microstrip feeding line at the two ends. The dimensions of the filter are chosen as follows: ?mm, ?mm, ?mm, and s = 0.143?mm. The substrate is 0.2?mm thick with a relative dielectric constant of 12.9. For the excitation, two ports at the beginning and at the end of the microstrip line are used. The patch on the dielectric substrate disturbs the electric and magnetic field distributions of the filter; the fields around the geometry are governed by Maxwell’s equations. The transmission coefficient of this planar filter is a function of the design variables and can be written as , where represents fitness function and will be described in detail later. The filter structure has six variable dimensions for controlling the transmission performance, which are the widths and the lengths of the patches. The variable dimensions are optimized within the ranges given below: To maintain the three-patch structure, the following is the restraint condition: Equations (7) and (8) can be seen as the boundary of the solution space. For the three GA-based optimization algorithms, each possible filter shape is encoded into an individual, in which each geometric parameter is represented as a gene. The desired low-pass filtering characteristic has a cutoff frequency of 12?GHz. The fitness function is written as where is the total number of samples over the frequency range of interest (from 2?GHz to 22?GHz), is the transmission coefficient at the sample, and for the frequency range from 2?GHz to 12?GHz and for the frequency range from 12?GHz to 22?GHz.
Here, the commercially available software CST Microwave Studio is used for the full-wave analysis and the calculated transmission and reflection coefficients are gathered for the fitness evaluation. This software employs the FDTD method coupled with the perfect boundary approximation and the finite integration technique [32]. The robustness and accuracy of using CST Microwave Studio for electromagnetic designs have been demonstrated for a wide range of the electromagnetic spectrum.
Under the same predefined design variables and fitness function, the optimization was performed using the GA, FCGA, and NN-FCGA with the same empirical optimization parameters defined in the previous section. We note that the same initial population was used for all three cases. Figure 4 shows the best fitness values plotted as a function of the generation number for the GA, FCGA, and NN-FCGA. These convergence curves are averaged over three runs of each algorithm. As can be seen in Figure 4, the NN-FCGA and FCGA have nearly the same convergence speed, with both speeds being much faster than the convergence speed of the traditional GA. After convergence, the NN-FCGA can obtain superior results relative to the GA, but slightly worse than the FCGA. Figure 5(a) shows the transmission coefficient of the optimized low-pass filters; the dimensions of these optimized filters are summarized in Table 1.
(a)
(b)
It is clearly seen that the GA shows better results than the original design [31], while the NN-FCGA and FCGA show further improvement in suppressing the slobs along the lower frequencies near the cutoff frequency. It is also observed that the optimized filter shapes of the NN-FCGA and FCGA are quite similar (see Figure 5(b)) whereas the shapes differ from that of the GA. This implies that the NN-FCGA’s global search ability approaches that of the FCGA.
We now consider the issue of computation time. The execution time for the FDTD simulation for a particular filter shape is about 1.5 minute on a single processor. The training process and fitness evaluation for the NN take about 3 minutes and less than 0.1 second, respectively. Therefore, the computation time (on an HP Pentium IV-3.06 GHz PC with 3.37?GB RAM) for each generation of the GA and NN-FCGA are 60 minutes and 63 minutes, respectively; it is worth noting that both algorithms yield nearly the same computation time. In contrast, the computation time for each generation of the FCGA depends on the number of family members whose fitness value is required to be evaluated and it would linearly increase with increasing the number of children. The total computational time for GA, NN-FCGA, and FCGA are 60, 63, and 600 hours, respectively.
The role of NN in the NN-FCGA hybrid method is to act as a nonlinear function approximator, which can estimate the fitness value of a particular filter shape and replace the time-consuming FDTD simulation. The NN is first trained using the FDTD simulation results obtained from the initial population. Then, in each generation, the surviving individuals, which are chosen from the numerous families by the NN, are recalculated using the FDTD simulation. The results are then sent back to the NN in order to retrain the network model, thus yielding a more accurate one. After the network has been trained, the fitness value for an arbitrary individual can be readily obtained by the NN in a few microseconds. Since the NN takes only a small amount of time for training its network model and fitness evaluation, the NN-FCGA can save a large amount of computation time from fitness evaluation in the family competition process. This allows us to apply the family competition strategy to practical designs with complex and time-consuming models for the fitness evaluation. In short, as compared to the FCGA, the NN-FCGA can save a great deal of computation time, while maintaining the convergence speed and optimization performance. Therefore, the NN-FCGA appears to be the most efficient one among the three algorithms.
Moreover, this NN model can help us get more insight into the influence of each physical parameter on the fitness value. To illustrate this idea, we used the NN-FCGA optimized filter as an example to study the dependence of the fitness value on the width of the metallic patches . The dimensions of the NN-FCGA optimized filter are obtained from Table 1. Since our goal is to study the influence of the dimensions , , and on the fitness values, one specific parameter is varied at a time, while the other parameters remain the same as those listed in Table 1. Figure 6 provides the comparison of the parameter-sweep results performed by the NN and the full-wave analysis. The fitness values evaluated by the NN are found to be comparable to those evaluated by the full-wave analysis, especially in predicting the trend of changes in the fitness values when the physical parameters are varied. It would seem that increasing the values of and and decreasing the value of are preferable in improving the performance of the filter; this also reflects that the previous optimized shape, where and are very large and is small, is fairly reasonable.
We also note that, in the relative high fitness region, especially for the case, the discrepancies between the results given by the NN and the full-wave analysis are found. This may be attributed to the nonuniform distribution of chromosomes in the search space. During the GA’s evolution process, chromosomes with high fitness values are largely filtered and thus the sampling process for the NN model is not uniform. Therefore, due to the insufficient training samples, the fitness estimation for the relative high fitness region is not very stable. In addition, the performance of the NN used in this paper is not optimized, which is also a possible reason to cause such a discrepancy. However, the tendencies yielded by the NN agree quite well with the tendencies yielded by the full-wave analysis. Since the selection algorithm used here is the “ranking” method, this discrepancy will not affect the final results; provided that the probability of being selected for each chromosome depends on its “rank” but not its exact fitness value. More effective training algorithms or multiple-layer NN are suggested to be used to, wholly or partially, eliminate the discrepancy between the NN and the physical simulator. This is however considered as our future work.
3.2. Electromagnetic Absorber Design
The anechoic chamber plays a very important role in the experimental characterization of antennas and scatters. Typically, the walls of the anechoic chamber were coated with a lossy material to simulate a free-space environment and backed with a conducting ground to avoid the outside interference. To avoid the reflections at the air-absorber interfaces, the front surface of the absorber is designed using a periodic and well-shaped structure. In order to achieve better absorbing performance, the optimization design of the absorber structures has been an active research topic for the past several years [6, 14]. In the following, we will study the three GA-based optimization algorithms by applying them to the polygonal absorbers. Figure 7 shows the geometry of the polygonal absorber, which has a period of length along the -axis and extends to infinity along the -axis. Each grating in the absorber is symmetric with respect to the -axis. A standard magnetic radar absorbing material [33] (MAGRAM) is used for the shaped coating region of the absorber and backed by a perfect electric conductor (PEC). The height h and period d of the coating are set to 8 mm and 2 mm, respectively. The period d of the absorber is divided into 2(N - 1) segments of equal length; the coordinates of each point are given by In this case, the following restraint must hold: Each possible absorber shape is encoded into an individual for the GA-based optimizations. The height of the coating at each point is represented as a gene of an individual. We set where, due to the symmetric constraint, only ten genes (a half region of the absorber) is encoded into an individual. Our optimization goal is to minimize the reflection coefficient from the absorber at normal incidence for the (1) TM polarization ( out of the page), (2) TE polarization ( out of the page), and (3) TM + TE polarization over a frequency range from 8?GHz to 18?GHz. In these cases, the fitness function is written as where is the total number of samples over the frequency range of interest and is the reflection coefficient at the sample. For the polarization case, is the average of the reflection coefficients from the TM polarization and TE polarization at the sample.
Figures 8(a)–8(c) show the best fitness values plotted as functions of generation numbers using the GA, FCGA, and NN-FCGA for the TM polarization, TE polarization, and polarization absorbers, respectively. For each algorithm, the convergence curves are averaged over three runs with the same initial population and empirical optimization parameters. As can be seen in Figure 8, the convergence speed of the NN-FCGA lies in between the FCGA and GA, while, after convergence, the NN-FCGA’s best fitness is close to that of the FCGA. It is clear that, as compared to the GA, both the NN-FCGA and FCGA show faster convergence speeds as well as the improved fitness values at the end of optimization process. Figures 9(a)–9(c) show the optimized reflection coefficient (in dB) for the TM polarization, TE polarization, and polarization absorbers, respectively. The reflection coefficient in Figure 9(c) is the averaged reflection coefficient of the TM and TE polarization. Also, the reflection coefficient for a planar absorber made of an 8?mm thick thin-film of MAGRAM material is shown for comparison. The planar absorber shows an averaged reflection coefficient of –6.6?dB. It is seen that, with the GA-optimized arbitrary shape, the averaged reflection coefficient can be greatly minimized. However, the FCGA and NN-FCGA show further improvement in the absorbing performance. Although the NN-FCGA’s absorbing performance cannot exceed that of the FCGA, it shows distinct improvement compared to the GA. Figures 9(d)–9(f) show the optimized absorber shapes for the TM polarization, TE polarization, and polarization. We can find that the optimized shapes for the TM + TE polarization (Figure 9(c)) absorbers can be seen as a combination of the optimized shapes for the TM polarization (Figure 9(a)) and TE polarization (Figure 9(c)). This shape serves as a compromise between the absorbing performance of the TM and TE polarizations. As we compare the resulting shapes obtained from these three optimization algorithms, it is observed that the shapes obtained from the NN-FCGA and FCGA are quite similar, but differ from that of the GA. This implies that the NN-FCGA’s optimum search ability approaches that of the FCGA. Next, as we consider the computation time for the TM and TE polarization absorbers, the execution time of the FDTD simulation of a particular absorber is about 1 minute on a single processor and the computation time for each generation of the GA, NN-FCGA, and FCGA are 40 minutes, 43 minutes, and 6.67 hours, respectively. Therefore, the total computation time of numerical optimization for the GA, NN-FCGA, and FCGA are about 40, 43, and 400 hours, respectively. We note that the computation time for the polarization absorber should double the time given above because both polarizations are needed to be calculated by the FDTD simulation. Next, we randomly generate ten individuals and compare the fitness values evaluated by the NN model and FDTD simulation. Here, the TM polarization absorber is used to test the NN’s approximation ability. The averaged reflection coefficients (fitness values) evaluated by the NN and FDTD simulation for the arbitrary absorber shapes are shown in Figure 10. It is found that the NN’s approximation results agree quite well with the FDTD simulation results.
(a)
(b)
(c)
(a)
(b)
(c)
(d)
(e)
(f)
4. Conclusions
To summarize, two different types of electromagnetic designs, the low-pass planar filter and the electromagnetic absorber, are optimized using three GA-based optimization algorithms: the GA, the FCGA, and the proposed NN-FCGA. Among them, the GA has the worst optimum searching ability and the FCGA has the best optimum searching ability. However, the FCGA is extremely time-consuming, thus limiting the number of practical applications. There are several advantages that make the NN-FCGA a good alternative. First, the NN-FCGA can further improve the optimization results of the traditional GA, while the additional numerical optimization time is almost negligible. Although the NN-FCGA’s optimization results may not exceed those obtained using the FCGA, the NN-FCGA’s numerical optimization time is several times shorter than that for the FCGA. Moreover, a well-trained NN model can be obtained during the NN-FCGA’s optimization process. This model can help us get more insights into the influence of design parameters on the overall device performance. The design criteria can also be easily applied to any electromagnetic or other physical system whose objective characteristics can be known to the design parameters and the algorithm is thus completely general.
Acknowledgments
The authors would like to thank Professor C. T. Sun of Department of Computer Science, National Chiao Tung University, Taiwan, Dr. J. H. Tsai of Department of Simulation and Modeling, National Nano Device Laboratories, Taiwan, and Professor T. J. Yen of Department of Material Science and Engineering, National Tsin Hua University, Taiwan, for their fruitful discussion on evolutionary computing and full-wave analysis using CST Microwave Studio.