#### Abstract

Many genetic algorithms (GA) have been applied to solve different NP-complete combinatorial optimization problems so far. The striking point of using GA refers to selecting a combination of appropriate patterns in crossover, mutation, and and so forth and fine tuning of some parameters such as crossover probability, mutation probability, and and so forth. One way to design a robust GA is to select an optimal pattern and then to search for its parameter values using a tuning procedure. This paper addresses a methodology to both optimal pattern selection and the tuning phases by taking advantage of design of experiments and response surface methodology. To show the performances of the proposed procedure and demonstrate its applications, it is employed to design a robust GA to solve a project scheduling problem. Through the statistical comparison analyses between the performances of the proposed method and an existing GA, the effectiveness of the methodology is shown.

#### 1. Introduction and Literature Survey

The combinatorial optimization involves problems in which their set of feasible solutions is discrete or can be reduced to a discrete one, and the goal is to find the best possible solution. In areas such as routing, task allocation, scheduling, and so forth, most of the problems are modelled in the form of combinatorial optimization problems.

Due to the NP completeness of many combinatorial optimization problems, they are quite difficult to be solved analytically, and exact search algorithms such as branch and bound may degenerate to complete enumeration, and the CPU time needed to solve them may grow exponentially in the worst case. To practically solve these problems, one has to be satisfied with finding good and approximately optimal solutions in reasonable, that is, polynomial time.

In recent decades, researchers have developed evolutionary algorithms to solve the combinatorial problems with practical sizes. Evolutionary algorithms (EA) form a class of search methods that work by incrementally improving the quality of a set of candidate solutions by variation and selection (Eiben and Smith [1]). Genetic algorithms suggested by Holland [2] is an evolutionary algorithm based on the principles of survival of the fittest and adaption originated of Darvin’s evolution theorem. According to evolution theorem, new offspring is generated by joining old generations. An offspring with more fitness to environment is more able to survive and reproduce. The basic GA idea is as follow: the genetic pool of a given population potentially contains the solution, or a better solution, to a given adaptive problem. This solution is not “active” because the genetic combination on which it relies is split between several subjects. Only the association of different genomes can lead to the solution. This algorithm increases the fitness of solutions by the simulation of evolution process. For more details, see Goldberg [3] and Deb [4].

The two major steps in applying any GA to a particular problem are the specification of the representation and the evaluation (fitness) function (Deb [5]). These two items form the bridge between the original problem context and the problem-solving framework. The “chromosomes” encode a group of linked features of the problem, and the “Genes” encode the activation or deactivation of a feature. When defining a GA one needs to choose its components, such as variation operators (crossover, mutation, and recombination) that suit the representation, selection mechanisms for selecting parents and survivors, and an initial population (Ng and Perera [6]). Each of these components may have parameters, for instance, the probability of mutation, the problem of crossover, or the population size. The values of these parameters greatly determine whether the algorithm will find a near-optimum solution and whether it will find such a solution efficiently.

Evolution in GA is partly made at random, and the other part is based on both the behaviour of the applied patterns in component of GA and the setting desired values of the GA parameters. Therefore, the efficiency of a GA extremely relates to the selection of good patterns and tuning of its parameters. Many researchers have tried to optimize their GAs so far. In total, setting GA parameters are commonly divided into two cases: parameter control, and parameter tuning. In case of parameter control the parameter values are changing during a GA run. This requires initial parameter values and suitable control strategies, which in turn can be deterministic, adaptive, or self-adaptive (Eiben et al. [7]). For the case of parameter tuning, much attention has been focused on finding the theoretical relationship among these parameters. Schwefel [8] developed theoretical models for optimal mutation rates with respect to convergence and convergence rates in the context of function optimization.

De Jong and Spears [9] presented theoretical and empirical results on interacting roles of population size and crossover in genetic algorithm. As other approaches in optimization of GA parameters, Friesleben and Hartfelder [10] proposed a meta-GA approach in which both GA components and GA parameters are tuned. They demonstrated the importance of the right choice for the GA operators. Samples et al. [11] showed how parameter sweeps can be used for robustness and correlation analysis. Preuss and Bartz-Beielstein [12] embedded sequential parameter optimization in a wider framework of experimental EA design. However, majority of parameter tuning are based on design of experiment (DOE). Bagchi and Deb [13] proposed a DOE approach (a factorial design) to calibrate the parameters of GA using pilot GA runs of relatively short length. As other implementations of DOE for parameter calibration, Ruiz and Maroto [14] and Ruiz et al. [15] used DOE to select the best GA component while previous works just considered DOE for setting parameter values. In addition, Saremi et al. [16] applied DOE to find the effects of individual parameters (factors) as well as to reveal any significant interaction among parameters. In the general field of experimental design, a paradigm shift that emphasizes a low cost of tuning over the performance of optimal parameter values was due to Taguchi and Wu [17]. For more studies on optimization of GA, parameters refer to François and Lavergne [18], Czarn et al. [19], Costa et al. [20, 21], and Lobo et al. [22].

While in the previous researches, there has been less attention in selecting the optimum or near the optimum pattern for the components of a GA and tuning its parameters simultaneously, in this paper, we attempt to introduce a new approach in which at first the effects of the GA components and parameters as well as their interactions are statistically analyzed. Then, in the second step, the optimal values of the parameters are discovered. Whereas the previous works have just assessed the effects of parameters while they were adjusted on some discrete points, in the present work we, search a continuous interval in order to find the optimal values of the parameters.

The organization of the rest of the paper is as follows. The description of the proposed methodology is given in Section 2. Section 3 contains the application of the proposed procedure in a project scheduling problem. The performance evaluation and statistical analyses of the proposed method applied to a project scheduling problem come in Section 4. Finally, Section 5 concludes and suggests some future areas of research.

#### 2. The Proposed Methodology

A genetic algorithm is designed as a combination of patterns in encoding, generation, selection, joining, replacement, and stopping criteria. Although different patterns have been proposed to encode and to generate initial population, the relationships between the problem variables and the condition of the search range play an important role in designing the patterns. Nevertheless for other components of a GA, while a desirable combination is usually unknown, almost all existing patterns can be generally used. Furthermore, it seems that to design a robust GA, one needs to employ the following three phases: Phase 1: designing different patterns to create the, algorithm Phase 2: finding the significant combination of, and the designed patternsPhase 3: tuning the parameters of the significant combination.

In the next subsections, we first review some of the most commonly used patterns of the GA components and then demonstrate the existing methods in the last two phases. At the end of this section, the proposed methodology will be described.

##### 2.1. Phase 1: Designing Different Patterns to Create the Algorithm

In this section, we review some of the most commonly used patterns in GA components.

###### 2.1.1. The Encoding Patterns

In a typical optimization problem, there exists a wide discrete/continuous range of points named search range, and the goal is to find a point that leads us to the optimal solution. An essential characteristic of GAs is the encoding of the variables that describe the problem. There are many encoding patterns. Some patterns such as binary and gray encode the problem by a set of ones and zeros, and some others such as random keys, permutation, and real value encodings use numeric values to represent the chromosomes. For a review of encoding patterns, each with specific characteristic and capability, see (e.g. Jenkins [23], Hajela [24], Reeves [25]). Among these patterns, the real encoding has been shown to have more capability for complex problems (Andrzej [26]).

###### 2.1.2. The Selection Patterns

Some of the most prevalent of the chromosomes’ selection patterns proposed so far are based on the chromosomes’ fitness and are the roulette selection (Michalewicz [27]) and the tournament selection (Back et al. [28]). Some other patterns do not deal with the chromosomes’ fitness such as random selection in which the chromosomes are randomly selected and unlike selection in which all chromosomes attend the joining processes. A comparison of the selection varying schemes has been performed by Goldberg and Deb [29].

###### 2.1.3. The Crossover Patterns

One of the most important components of a GA is the crossover operator. There are two broad classes of the crossover operation, namely, the point crossovers (Deb [5]) and the uniform crossover (Syswerda [30]). In a point crossover, the string of the parents’ chromosomes is randomly cut on one or more points. Then by replacing the existing genes between the cut points of the parents’ chromosomes, a new child is created. However, in a uniform crossover, any genes of both parents’ chromosomes have a chance to attend the child's chromosome.

Traditionally, GAs have relied upon point crossovers. However, there are many instances in which having a higher number of crossover points is beneficial (Eshelman et al. [31]). Furthermore, perhaps the most surprising result is the effectiveness of the uniform crossover. Uniform crossover produces, on average, crossings on string length (Spears and De Jong [32]). In addition to empirical investigations, considerable effort has been directed towards the theoretical comparisons between different crossover operators (De Jong and Spears [33]). Nonetheless, these theories are not sufficiently general to predict when to use crossover, or what form of crossover to use.

###### 2.1.4. The Mutation Patterns

Another important operator of a GA is mutation that acts like an insurance policy against premature loss of important information (Goldberg [3]). Since the mutation is a structural change in a set of candidate genes of a chromosome, it is directly related to the designed encoding pattern. For example, in a binary encoding, mutation can be defined as changing the value of a gene from one to zero. However, two common viewpoints exist to perform the mutations that are not related to the encoding pattern. In the first one, a mutation probability is assigned to each child. Whenever the mutation is going to be performed, it just has to decide to mutate all of the candidate genes of a child or none. In the second one, a mutation probability is assigned to each candidate gene. Then, through each mutation operation, the algorithm faces a number of decision-making steps that is equal to the number of the candidate genes. In fact, for each candidate, the algorithm has to assess whether the mutation is performed or not. Therefore, it usually occurs that some of the candidates have been mutated and the rest stay without any change.

###### 2.1.5. The Replacement Patterns

In a GA, the children of the old generation are replaced with the newly generated children. A variety of methods have been proposed in the literature for this task. In one of these methods called (), in each generation the existing chromosomes generate new children and then through -generated children new children are chosen for the next generation. It is obvious that in this method must be greater than . Otherwise, after some generations the population size may become zero. In the second method named (), the existing chromosomes in each generation create new children and then through these available chromosome, children are chosen. Michalewicz [27] experimentally showed that the () method performs better than the () method in terms of the solution quality. In another replacement method named Elitism, few of the most fitted children are directly reproduced to the next generation, and the other members of the new generation are created based on the genetic operators. This replacement method has the potential of converging to the global optimal effectively. When this method is used, the number/percentage of the reproduced children can be considered as one of the GA parameters. Finally, another replacement strategy is called preselection in which if any child shows better fitness than its parents, it is replaced with the one with better fitness (Goldberg [3]).

###### 2.1.6. The Stopping Patterns

The stopping criteria are usually of two types. The first one (such as either reaching a predefined number of iteration or a predefined iteration time of running GA) is called the passive stopping criterion in which the algorithm independently of the obtained results is stopped. The second one (such as either reaching a unique solution in some sequential generations or the difference between the average fitness of some sequential generations is low) is called the sequential criterion in which stopping the GA depends on the quality of the obtained results. Both of these criteria may be treated as the GA parameters.

In summary, some of the predetermined patterns along with their parameters and symbols used to describe them are given in Table 1. It should be noted that selecting the significant combination of the patterns among more designed patterns would lead to more accurate basis of designing a robust GA. Many of these patterns have the same structure such that they can be transformed to each other during their designing phase. In this case, the pattern creation and designing do not need too much times and efforts.

##### 2.2. Phase 2: Finding the Significant Combination of the Designed Patterns

While the performances of a GA (responses) are affected by some controllable factors such as encoding, selection, crossover, mutation, replacement, and stopping patterns, as well as uncontrollable factors in which their different combinations would result in different performances, design of experiments can be implemented to study the effects of the input factors and their interactions on the system performances and delineate which factor(s) has (have) the most effect(s) on the response(s). Furthermore, when the study involves two or more factors, factorial designs are generally the most efficient way of delineating the significant factors (Montgomery [34]).

In designing a robust GA, the effects of different patterns and parameters along with their interactions can be analyzed using a factorial design such as a full factorial design (a factorial), or a fractional factorial design, in which is the number of factors and represents a fraction. If the results of the analysis of variance table of such designs indicate statistical significance, then a post hoc analysis may be performed to assess the important factor(s). The least significant difference (LSD) and the Duncan tests are among the tests that compare all paired factor means and delineate the important factor(s) (Montgomery [34]). Either of these tests as well as others may be employed to find the important patterns affecting the efficiency of the designed GA.

##### 2.3. Phase 3: Tuning the Parameters of the Significant Pattern Combination

Response surface methodology (RSM) is a collection of statistical and mathematical techniques useful for modelling and analysis of problems in which responses affected by several variables, and its goal is to optimize these responses. In most of these problems, the relation between the response and the factors (independent variables) is not definite, requiring to first estimate this relationship. To do this, if there exists no curvature in the response surface, one can employ the first order model; otherwise, a model of higher order (usually a second order model) should be used. Then, a level combination of the factors should be found such that the optimal value of the response is reached. To do this, the deepest ascent method (for maximization problem) or the deepest descent method (for minimization problem) are employed to move on a path with the most gradient that makes the most increase or decrease in the objective function. A comprehensive survey of RSM refers to Myers and Montgomery [35].

In order to tune the parameters of the designed GA, one may use RSM. If the relation between the algorithm efficiency and the important GA parameters is linear, then depending on the constraints (time and cost) on the size of experiments either the factorial or fractional factorial design is used. In case the relation is nonlinear, the central composite design (CCD) or other desirable designs can be applied to estimate the response function. The response surfaces for designing a robust GA may be defined as (a) minimization of the relative deviation percentage (RDP) obtained by employing the algorithm to the optimal result and (b) minimization of the algorithm runtimes or optimization of another goal set by the algorithm designers. In case of multiple goals, the multiobjective optimization methods such as goal programming (GP), fuzzy goal programming (FGP) (Narasimhan [36], Tiwari et al. [37]), or the other known methods can be applied.

The present methodology can be carried out independent of the selected encoding pattern. This is due to the fact that the patterns work independent of encoding as they use the fitness of chromosomes. In other words, regardless of the encoding patterns, we need to have a method to calculate the fitness of chromosomes. There is also a similar consequence for the replacement patterns. They all are either completely free of encoding type or in some situations such as Elitism they are only associated with the fitness. For the most important part of GA, that is, the crossover pattern, since some characteristics (genes) of patterns are replaced through the point and the uniform crossovers, the methodology is applicable in all encoding patterns. The only part of GA that is associated with the encoding system is mutation. As explained in Section 2.1.4, there are two strategies regulating how the designed mutation is applied. In this case, since different patterns can be defined through the encoding type, the present methodology can be still applicable.

In the next section, a project scheduling problem is used to demonstrate the steps involved in the proposed methodology along with its possible implementation in practice.

#### 3. Application of the Proposed Methodology in a Project Scheduling Problem

Project scheduling problem is an important branch of combinatorial optimization problem in which the goal is optimization of one or some objectives subject to constraints on the activities and the resources. Due to its NP-Hardness structure, researchers have widely considered metaheuristics, particularly genetic algorithms, in order to solve it. Among these research works, different chromosome encodings and parameter settings have been introduced so far. For example, the priority-value based and priority rule-based representations were developed by Lee and Kim [38] and Özdamar [39], respectively. Hartmann [40] and Alcaraz and Maroto [41] used permutation-based encoding. Furthermore, Alcaraz and Maroto [41] compared different selection operators and showed the best is the 2-tournament operator in which two individual solutions are selected randomly from the current population, and the best fitted is selected as a parent-solution. Gonçalves et al. [42] presented a genetic algorithm for the resource constrained multiproject scheduling problem based on random keys for the chromosome encoding. To adjust the parameters, they investigated four factors: (1) the top percentage from the previous population chromosomes that are copied to the next generation (TOP), (2) the bottom percentage of the population chromosomes that are replaced with randomly generated chromosome (BOT), (3) the crossover probability (CProb), and (4) the population size. A small pilot study was performed by experimenting the combinations of these factors.

Resource investment problem with discounted cash flows introduced by Najafi and Niaki [43] is a branch of project scheduling problems in which the projects are mathematically formulated so that the resource levels and starting times of activities are the decision variables. Since the cash flows occur during the project, its goal is maximization of net present value of the project cash flows. This is an NP-hard model and cannot be easily solved. Najafi et al. [44] developed a model of this problem by minimal and maximal time lags and suggested a genetic algorithm to solve it. In their GA, a real-valued encoding has been used to represent the chromosomes. In this regard, each individual chromosome, , is a vector consisting genes where denotes the number of the project activities. Each position in the chromosome denotes the corresponding activity number, and the value of gene states the starting time of activity at the precedence feasible schedule of the chromosome. For example, consider a project whose network is depicted in Figure 1. The values located in the nodes indicates the activity numbers and the ones associated with the arcs connecting activity to activity display the minimum time lag between start to start activity as predecessor of activity . For the network shown in Figure 1, is a chromosome showing a real schedule for starting time of the activities.

Since Andrzej [26] showed that the real encoding has more capability for complex problems and the algorithm of Najafi et al. [44] has a real-value based representation for chromosomes, in this research, we decided to implement them as well. Shortly, the tournament selection pattern () was the choice of selecting the parents. Moreover, the crossover operation has been performed in a combination method involving two operators (one-point crossover () and uniform crossover ()). For mutation operation, assigning a probability to any chromosome () and for children replacement, the preselection method () has been selected. In addition, to stop the algorithm a combination of two criteria has been applied, a predefined number of iteration along with a number of sequence generations with the same good solution. All of the patterns used in their algorithm have been reached based on a trial and error method. They tested the algorithm on 180 test problems included 10, 20, and 30 nondummy activities with 1, 3, and 5 resources and compared their GA solutions with the ones obtained by solving the mathematical models of the problem.

##### 3.1. Phase 1: Designing Different Patterns

Since having more patterns are desired for a robust GA, in addition to the ones devised by Najafi et al. [44], we design more pattern combinations in the algorithm efficiency comparison study. For this purpose, one other selection pattern, the unlike selection (), assigning a probability to any candidate gene for mutation (), and the Elitism replacement method () are also employed. The tournament and the unlike patterns are the two levels of factor . Furthermore, both one-point and uniform crossover operators are considered in the study. While the one-point and the uniform crossover operators were considered as a combined factor in Najafi et al. [44] method, in this research they are the two levels of factor . The probability assignment to both the children and the genes are the two levels of factor . The two levels of factor are the preselection and the Elitism. Factor has two levels of the number of iterations (passive stopping criterion) and reaching a good solution in three consecutive generations (sequential criterion). Finally, factors , , and , each with two quantitative levels, denote the crossover probability, the mutation probability, and the local improvement probability, respectively. The levels of factor , , and are , , and , respectively. Table 2 shows the factor levels under study.

##### 3.2. Phase 2: Finding the Significant Combination

One run of the full factorial design of the 8 aforementioned factors requires experiments. Due to the time and cost, constraints suppose a design with smaller run numbers is desired. Whereas all of these factors contain 2 levels, a fractional factorial design is a good option. Even if factors with more than two levels exist in the experiment, either a higher-order fractional factorial or a general factorial design may be used. In this study, a fractional factorial design containing 16 runs with generators , , , and , is used to test the performances of the proposed methodology on the 30 test problems with known optimal solution.

For the analysis of variance, the relative deviation percentage (RDP) of the proposed GA solutions to the optimal solutions which is calculated according to (3.1) was used as the response.

Although the probability distribution of the response is not known, since 30 test problems is used in each of which the summation of the responses are used for different factor combinations, the central limit theorem implies an approximate normal distribution. Assuming that more than 2-way interaction effects are not significant, the analysis of variance of this experiment is given in Table 3. The results of Table 3 show that the main effects, along with the 2-way interaction effects, are statistically significant.

In order to find the specific significant effects, the Duncan's multiple range test as one of the ad hoc analysis methods results in the following ranking:

In which “” shows the better and “” shows the statistically better, and “−1” and “+1” of the superscripts denote the low and the high levels of the corresponding factor. Thus, the factor combination is known as the significant combination and , , and are known as the neighbour points of the optimal values of some important parameters of the GA that needs to be considered to be tuned in the next phase.

##### 3.3. Phase 3: Tuning the Parameters

The candidate parameters in the tuning process are population size (), crossover probability (), mutation probability (), and local improvement probability (). Table 4 shows the search ranges of these parameters. The lower and the upper points of the ranges are considered as the lower and the upper levels of these factors and coded as ±1. Furthermore, shows the number activities of the project.

In the tuning phase, a 2^{4} central composite factorial (CCF) design with 4 central points and 8 axial points of (,), (), (), and () including 28 combinations of the factor levels is employed. Table 5 shows the factor level combinations along with their corresponding responses.

The first response denotes the mean RDP, and the second response () represents the mean ratio of the proposed GA CPU-times to the required optimal CPU-times, where the optimal CPU-time of each problem is defined as the minimum time obtained in 28 executed levels.

The results of Table 5 are used to estimate both functions (accuracy of the solution) and (quality of the solution) using SAS software. These estimates are given in (3.3) and (3.4), respectively,

Tables 6 and 7 contain the analysis of variance results that show significant linear, quadratic, and interaction effects for both the solution accuracy and quality at 90% confidence level.

Since the goal is to find the parameters values such that both objective functions are simultaneously optimized, a bi-objective optimization problem is needed to be solved. The fuzzy goal-programming (FGP) technique transforms the multiobjective decision-making problem to a single objective using fuzzy set theory. In FGP, a membership function is defined for each objective () based on (3.5), in which is its objective value when it is optimized alone, and is its worst objective value considering the values of the variable set obtained in other objectives when they are optimized separately. The goal is to reach . However, since conflict may happen by the other goals, FGP tries to maximize the achievement degree of to resolve the conflict

Suppose to be the achievement degree of . The single objective function in FGP is then the maximization of the weighted sum of the achievement degrees () of the goals () subject to the existing constraints and the ones added for the membership functions (e.g., for objective ).

To solve this problem, we first obtain the pay-off table of the positive ideal solution (PIS) as shown in Table 8.

The membership functions of these two objectives can be obtained as follows:

Then, the model becomes

The optimal values obtained by the FGP method with and for and , respectively, are shown in Table 9.

#### 4. Performance Analyses

To compare the performance of the proposed optimized genetic algorithm, all 180 problems used in Najafi et al. [44] are applied by the new algorithm as well. Then, statistical tests are used to compare the results. The results of the comparison experiments on the test problems are shown in Table 10.

The following notations are used in Table 10. : Average relative deviation percentages of the optimized GA solution to the GA of Najafi et al. [44] solution.: Average runtime (in seconds) required to obtain the solutions by GA of Najafi et al. [44].: Average runtime (in seconds) required to obtain the solutions by the optimized GA.: Average relative-deviation-percentages of the optimized GA runtime to the GA of Najafi et al. [44] runtime.

These results of Table 10 show that the optimized GA performs better than the GA of Najafi et al. [44] solutions (on the average 3.6% better) with less amount of required CPU time (on the average −14%). Furthermore, two tests of hypothesis are performed to prove the statistically better performances of the proposed methodology. The results of the tests that are given in Tables 11 and 12 show that the performances of the proposed optimized GA is better than those of the GA of Najafi et al. [44].

#### 5. Conclusion

The genetic algorithm is known as one of the most robust and efficacious methods to solve combinatorial optimization problems and has been widely used in recent researches. Since different viewpoints suggested to design this algorithm and its parameters greatly affect the solution quality, in this research a methodology that contains three phases has been proposed to design an optimal genetic algorithm. The phases are designing different combinations of common viewpoints to create GA, selection of the significant combinations by design of experiments, and tuning the parameters using response surface methodology. This methodology was then applied to optimize a GA to solve a project scheduling problem. Statistical comparisons of the results obtained by the proposed GA with the ones from an existing GA to solve the project scheduling problem verified better performances of the new algorithm with less amount of required CPU time. The proposed methodology can be employed for other encoding patterns of GA in addition to other metaheuristic algorithms as future researches.