A Novel Tournament Selection Based Differential Evolution Variant for Continuous Optimization Problems

Abbas, Qamar; Ahmad, Jamil; Jabeen, Hajira

doi:https://doi.org/10.1155/2015/205709

Mathematical Problems in Engineering

On this page

Abstract Introduction Experimental Results and Discussion Conclusion References Copyright Related Articles

Special Issue

Artificial Intelligence and Its Applications 2014

View this Special Issue

Research Article | Open Access

Volume 2015 | Article ID 205709 | https://doi.org/10.1155/2015/205709

A Novel Tournament Selection Based Differential Evolution Variant for Continuous Optimization Problems

Qamar Abbas,¹Jamil Ahmad,²and Hajira Jabeen¹

Academic Editor: Yudong Zhang

Received19 Sept 2014

Revised02 Dec 2014

Accepted10 Dec 2014

Published04 Oct 2015

Abstract

Differential evolution (DE) is a powerful global optimization algorithm which has been studied intensively by many researchers in the recent years. A number of variants have been established for the algorithm that makes DE more applicable. However, most of the variants are suffering from the problems of convergence speed and local optima. A novel tournament based parent selection variant of DE algorithm is proposed in this research. The proposed variant enhances searching capability and improves convergence speed of DE algorithm. This paper also presents a novel statistical comparison of existing DE mutation variants which categorizes these variants in terms of their overall performance. Experimental results show that the proposed DE variant has significance performance over other DE mutation variants.

1. Introduction

Storn and Price [1] have proposed DE in 1995 which is a stochastic population based evolutionary algorithm to find global optima for any given function [2]. DE has been applied to variety of real life problems such as electrical power systems [3], control systems [4], electromagnetism [5], image processing [6], bioinformatics [7], signal processing [8], chemical engineering [9], and many more. Different state-of-the-art versions of DE algorithm like jDE [10], SaDE [11], JADE [12], EPSDE [13], and so forth are based on parameter selection, parameter adaption, strategy selection, and/or strategy adaption mechanisms. DE state-of-the-art proves to be very powerful and uses conventional DE strategies as a strategy selection pool/strategy adaption and/or uses parameter selection pool/parameter adaption. During the process of optimization, DE algorithm evolves population of potential solutions by exploring the entire search space overtime to locate the optima.

The DE algorithm mutation strategies are formed by the linear combination of existing population members. The trial vector and target vector forms the mutant vector in DE. DE mutation strategies can be formed by any combinations of current vector, random vector(s), better vector, and best vector. In any mutation strategy, the order, number, and name of vector(s) are very important. The behavior of DE algorithm is influenced by the selection of mutation strategy and crossover scheme along with their control parameters: mutation probability “” and crossover rate “CR.” As stated, there are many mutation strategies associated with DE algorithm but current literature does not show any concrete study where all of the popular mutation strategies are analyzed. It has been observed by the authors that some of the DE mutation strategies are more useful than the current dominating ones. Therefore, in this paper, a comparative analysis is also carried out to identify best performer DE mutation variants. Furthermore, this research also proposes a novel DE mutation variant. The proposed DE variant is based on the well known tournament selection criteria. The proposed DE mutation strategy is abbreviated as TSDE in this paper. To generate a mutant vector in DE algorithm, some parents will be selected through tournament selection criteria. The TSDE variant has the ability to increase the convergence speed since it uses the combination of best vectors and random vectors from the population. Tournament selection will be helpful to incorporate some diversity in TSDE since global best vector has less chance of selection as a parent. To generate a mutant vector, the proposed TSDE utilizes two best individuals instead of global best from the whole population which makes it more convenient to escape from the local optima problem. The most state-of-the-art DE algorithms focuses on basic DE mutation strategies and the incorporation of this proposed TSDE mutation variant will be helpful in improving the performance of DE algorithm.

The rest of the paper is organized into the following sections. Section 2 presents literature survey. In Section 3, the DE algorithm operators are discussed. In Section 4, the proposed TSDE variant is presented. Section 5 presents test functions and parameter setting details. In Section 6, experimental results along with discussion are given. Section 7 contains statistical analysis. Section 8 compares proposed TSDE with other well known heuristic algorithms. Finally, conclusion and future work are presented in Section 9.

2. Literature Survey

As stated above, many state-of-the-art versions of the DE algorithms are developed, for example, jDE, SaDE, JADE, and DEGL. Variations in the mutation strategies of DE algorithm have also been the result of intensive research in this area. Qu and Suganthan [29] have introduced the concept of ensemble restricted tournament selection in DE algorithm. They have selected a set of population members and calculated the Euclidean distance of population members from the current individual. The member having smallest distance from current individual is selected as a tournament winner and it became the current individual if its fitness is also better. Epitropakis et al. [30] have introduced two new mutation strategies that incorporate spatial information of neighbors to form new population individual. Their new mutation strategies “DE/nrand/1” and “DE/nrand/2” corresponding to two well known mutation strategies “DE/rand/1” and “DE/rand/2” are important part of the current literature. Qiu et al. [31] have introduced the concept of tournament selection criteria in DE algorithm. They have selected all individuals used in their mutation strategy by using the concept of tournament selection by taking a tournament of size 3. Their research work is applied on “DE/rand/1,” “DE/rand/2,” “DE/current to best/2,” and “DE/rand to best/2” DE mutation strategies which proved to be competitive to original DE algorithm and also enhanced jDE algorithm. Neighbourhood based mutation strategy for DE algorithm was introduced by Das et al. [32]. This strategy implements the concepts of local model that works on small neighbourhood of each individual while global model mutates each individual using the neighbourhood concept of entire population. The concept of neighbourhood is implemented on a commonly used DE mutation strategy “DE/target-to-best/1/bin.” Sutton et al. [33] have introduced the concept of focus search by imposing a selective pressure for DE mutation strategies. The selective pressure is implemented by using the concept of rank based selection of vectors used in generating donor vector in DE mutation. A bias is imposed on selecting a parent using rank based selection criteria that is based on the fitness of population members. Epitropakis et al. [34] have introduced the concept of proximity based mutation operator in DE algorithm. Their new framework utilizes the information of neighbours in evolving the population. Each population member is assigned a selection probability that is inversely proportional to the distance from mutated vector. Proximity based mutation is applied to “DE/rand/1,” “DE/best/1,” “DE/rand/2,” “DE/best/2,” “DE/rand-to-best/1,” and “DE/rand-to-best/2” mutation strategies. Research result shows the significance of proximity based mutation operator in DE algorithm.

DE algorithm has a number of conventional mutation strategies where the performances of these strategies may differ from each other. Zaharie [35] has analyzed the influence of crossover variant on the choice of control parameters of the DE algorithm. He has used “DE/rand/1” for different parameter settings of population size, mutation probability, and the crossover rate by binomial and exponential schemes. Babu and Munawar [36] have applied various DE strategies for the optimal design of shell and tube heat problem which performed better than “DE/rand” for the same problem. They have used different DE variants for the selection of optimal design using various control parameter settings. They conclude that “DE/best” strategies have better performance than “DE/rand” for shell and tube heat problem. Jeyakumar and Velayutham [24] present empirical performance analysis of DE variants for unconstrained global optimization problems where they have shown that binomial variants have superior performance compared to the exponential variants. Mezura-Montes et al. [37] have performed comparative study of DE mutation variants for continuous function optimization application. They found through the empirical study that “DE/best/1/bin” has produced good results. Ali et al. [38] have compared Mixed Strategies Differential Evolution (MSDE) with the five different DE variants for function optimization. They have shown that performance of MSDE is better than five variants of DE algorithm. Mutation operators are also used in other well known algorithms like PSO [39].

Qin et al. [11] introduced both control parameter adaption and strategy adaption mechanisms in DE algorithm (SaDE). In strategy adaption scheme, they used a strategy candidate pool of four conventional strategies: “DE/rand/1/bin,” “DE/rand/2/bin,” “DE/rand-to-best/2/bin,” and “DE/current-to-rand/1.” Each target vector generates a trial vector based on the learning period (LP) over previous generations based on the success rate. In parameter adaption, SaDE adjusts control parameter CR based on the median value of CR that is calculated based on previous CR values that have successfully generated trial vector. Zhang and Sanderson [12] introduced new parameter adaption method and have used “DE/current to pbest/1” conventional strategy that is based on “DE/current to best/1.” Mallipeddi et al. [13] introduced ensemble based crossover and mutation DE strategy (EPSDE) and their corresponding control parameter scheme. They have used a pool of different crossover and mutation strategies and a pool of values for each associated control parameter. Each target vector generates a trial vector based on the assigned strategy and the parameter values. Successful combination of the mutation and crossover strategy and associated parameters values are stored in the pool. EPSDE uses “DE/current-to-rand/1/bin” and JADE mutation strategy along with binomial and exponential crossover. Islam et al. [40] have introduced modified mutation strategy, a modification in conventional binomial crossover scheme, and a parameter adaption scheme in DE algorithm. They have used “DE/current-to-gr_best/1” and call it a less greedy version of “DE/current-to-best/1.” Their mutation strategy “DE/current-to-gr_best/1” uses best vector from population of individuals to generate the trial vector for each target vector. In pbest crossover mutant vector can swap p-top ranked individuals of current generation instead of current parent using binomial crossover scheme. In parameter adaption scheme scale factor adaption is based on the Cauchy distribution and crossover probability adaption is based on the Gaussian distribution. Wang et al. [41] have introduced composite DE (CoDE) variant. In this scheme, they have used a pool of three trial vector generation strategies and a pool of three parameter setting combinations. The trial vector strategies used are “rand/1/bin,” “rand/2/bin,” and “current-to-rand/1” while the parameter setting combinations are [, ], [, ], and [, ]. To generate a new solution in CoDE, each strategy is coupled with a randomly chosen parameter setting. Gong et al. [42] have introduced a new strategy adaptation mechanism (SaM) in their research work. They have combined SaM with JADE and named it as SaJADE. In this strategy, parameter is used to control the selection of any strategy from the strategy pool. They have chosen four various DE strategies “DE/current-to-pbest” without archive, “DE/rand-to-pbest” without archive, “DE/current-to-pbest” with archive, and “DE/rand-to-pbest” with archive to form a strategy pool.

Table 1 shows different binomial and exponential mutation strategies used with algorithm. Current literature shows variation in naming in these strategies, which is already indicated in Table 1; for example, Four different names are used in the literature for mutation strategy . Throughout this paper, denotes the target vector (or current vector), represents the trial vector, and as a mutant vector. In this paper, states the th random vector for th generation, will refer to th component of donor vector at th generation, states the best vector at th generation, and refers to current vector at th generation.

3. DE Algorithm

DE algorithm has three different parameters: a population of size NP, crossover control parameter CR, and difference vector amplification parameter . Each population member in DE is represented as a -dimensional parameter vector. In DE algorithm, population is initialized randomly which is supposed to cover the entire search space. Each vector in the DE is represented by , where and is generation number. New offspring in DE algorithm are generated by mutation, crossover, and selection operators. Three different terminologies of vectors donor vector, trial vector, and target vector are used in DE algorithm. Donor vector is a vector that is created in the mutation operation, trial vector is created in the crossover operation, and target vector is the current vector of population. The detail of these operators with index , is as follows.

Mutation. In mutation operation, mutant vector also called donor vector is created. Donor vector of th population member is calculated by adding the weighted difference of two vectors to third vector:where random indices , , and is mutation probability parameter.

Crossover. DE crossover strategies control the number of inherited components from the mutant vector to form a target vector. Binomial and exponential are main crossover schemes. The DE crossover rate parameter (CR) influences the size of perturbation of the base (target) vector to ensure the population diversity [43]. Following are the binomial and exponential crossover schemes.

Binomial Crossover. In crossover operation of DE algorithm, a trial vector is formed. In binomial crossover scheme, the trail vector is generated by the following equation: where is a randomly chosen integer in the range , is a random number in , and is the donor vector. is crossover control parameter in the range . Due to the range of , is always different from and index , .

Exponential Crossover. In exponential crossover scheme, the trail is created as follows:where , , and denotes the modulo function with modulus . The starting index is chosen at random from . is also a randomly generated number from . The parameters and are regenerated for each trial vector .

Selection. In DE algorithm, new population members are formed using selection operation. Selection operator uses the greedy approach by comparing fitness of trial vector with the fitness of target vector ; the vector having best fitness is selected as a member of new population. For selection, the following equation is used:where fitness function calculates the fitness value of objective function.

There are several DE algorithm mutation variants/strategies that are formed by the linear combination of existing population members. The trial vector and target vector form the mutant vector in DE. Throughout this paper, denotes the target vector (or current vector), is the running index, represents the trial vector, and is a mutant vector. In DE algorithm, different mutation schemes are used to create the trial vector by using any combination of current, best, and random vectors. The behavior of DE algorithm is influenced by the selection of mutation strategy and crossover scheme along with their control parameters: mutation probability “” and crossover rate “CR” [44].

Difference vector (DV) is the difference of two mutating vectors that is used to form offspring in the population [45]. To form the mutant vector in DE, some researcher uses a random value as a coefficient multiplier with the first difference vector [1] and mutation probability “” as a coefficient multiplier with the other difference vector(s) [17]. Some researchers have used only “” as a coefficient multiplier with the difference vector(s) to form the mutant vector [44]. To reduce the number of control parameters of DE algorithm, we use [28]. A list of binomial and exponential mutation strategies that are used in this research is given in Table 1.

4. Proposed Tournament Selection Based DE (TSDE) Variant

Tournament selection is one of the famous selection approaches used in genetic algorithms [46]. The parameter associated with this scheme is the size of tournament that selects number of individuals to participate in the competition of tournament. To decrease the risk of premature convergence, the loss of diversity should be kept as low as possible in the population [47]. To enhance the performance of population based algorithms, various researches have used tournament selection in their research work [48, 49].

The DE algorithm sometimes faces the problem of slow and/or premature convergence [32]. Exploration and exploitation are very important aspects that are helpful in improving the convergence acceleration and solution quality of evolutionary algorithms [50]. The mutation strategies “DE/rand/1” and “DE/rand/2” are helpful for exploration [11] and “DE/best/1,” “DE/rand-to-best,” and “DE/best/2” are more helpful for exploitation than exploration due to less population diversity [14]. To balance the exploration and exploitation ability of proposed TSDE variant, both random and best vectors are used.

A novel DE mutation strategy based on the selection of parent is introduced in this research work. The new mutation strategy utilizes the knowledge of some best performing vectors and some random vectors in creating the mutant vector. Like mutation strategies “DE/rand/1” and “DE/rand/2,” the proposed mutation strategy uses random vector as a base vector along with two difference vectors. To generate a mutant vector in DE, each difference vector utilizes the knowledge of one distinct best vector that will be helpful in incorporating diversity and improving convergence in the proposed mutation strategy. The two difference vectors and two distinct best vectors selected through tournament selection criteria will be helpful in balancing the exploration and exploitation.

To minimize the loss diversity, a tournament of small size is selected so that less fit individuals may have a chance of selection as a parent. TSDE generates two best vectors using tournament selection mechanism. The selection of parents is based on the tournament selection criteria by taking a tournament of size 3 of randomly selected population members. The equation of the proposed variant TSDE is as follows: The equation of this variant uses two best vectors and and two random vectors and that are selected from the current population. Vectors and are selected by using tournament selection mechanism by taking tournament of any size, say . Vector is the best individual from first tournament and vector is the second best individual from the second tournament. The vectors used in each tournament are selected randomly from the current population.

The proposed TSDE variant has the ability to increase the convergence speed since it uses the combination of best vectors and random vectors from the population. Tournament selection may incorporate some diversity in TSDE since global best vector has less chance of selection as a parent because tournament of small size is used instead of whole population. Selection of two best individuals instead of global best from the whole population makes it more opportune to escape from the local optima problem that accordingly increases the solution quality and convergence speed of DE algorithm. The most state-of-the-art DE algorithm focuses on basic DE mutation strategies and the incorporation of this proposed mutation strategy will be helpful in improving the performance of DE algorithm.

The proposed mutation strategy has the following features.(1)A random number will be used as a base vector since mutation strategies “DE/rand/1” and “DE/rand/2” use random base vectors that maintain better exploration ability but have slow convergence speed [11].(2)Although DE mutation strategies “DE/best/1,” “DE/rand-to-best,” and “DE/best/2” have fast convergence [14], these are biased towards global best value that may have a chance to stick in local optima problem [11]. So to improve the convergence speed of proposed TSDE, two best vectors (best₁, best₂) are selected using tournament selection criteria.(3)The equation of proposed mutation strategy is where is scaling factor of each difference vector. The motivation of this mutation strategy is an existing DE variant “DE/rand to best/1” [23, 51] that places perturbation of random vector towards global best vector and contains two difference vectors with equation Two best vectors (best₁, best₂) in TSDE will be helpful in escaping local optima problem and will be helpful in improving the solution quality.(4)Two difference vectors are used where each difference vector utilizes one best vector selected through tournament criteria, since mutation strategies having two difference vectors (DV) produce better perturbation mode than one difference vector mutation strategies [41].

The members of tournament are selected randomly which almost makes it equal chance of selection of higher or lower performing individuals from the population. This variant maintains randomness since members of the tournament are selected randomly and we select the best performing amongst the tournament to ignore the poor performing individuals. The proposed variation is simple since it uses two best individuals selected using the tournament selection technique and other random vector selected randomly from the current population. The performance of TSDE is compared with other DE variants in Section 6.

A graphical representation of proposed mutation strategy is shown in Figure 1, where mutant vector is generated using four vectors, , , , and , shown in filled circles and directed arrow shows difference vector.

Algorithm 1 shows the pseudocode of the proposed DE algorithm (TSDE). The proposed method is also implemented through computer simulation which will be discussed in a later part in the paper.

(1) Generate the initial population for generation and randomly initialize
each population member where
(2) FOR to NP
Calculate fitness for each population member
END FOR
(3) WHILE the stopping criterion is not true
/^*Start of TSDE vectors selection ^*/
Step 3.1. TSDE vectors selection
FOR to number of TSDE vectors
FOR to Tournment_size
Select th tournament member with its fitness randomly from current population
END FOR
Select best of best member from the current tournament as
Return member index to be used as one of TSDE vectors in proposed mutation strategy
END FOR
/^*End of TSDE vectors selection ^*/
Step 3.2. Mutation Step
FOR to NP
For the th target vector generate a donor vector with the
specified mutation strategy (From Table 1 strategies or proposed TSDE strategy)
END FOR
Step 3.3. Crossover Step
FOR to NP
For the th target vector generate a trial vector with the
specified crossover scheme (Equation (2) or Equation (3))
END FOR
Step 3.4. Selection Step
FOR to NP
Evaluate the trial vector against the target vector with fitness function f
IF , THEN ,
IF , THEN ,
END IF
END IF
END FOR
Step 3.5. increment generation number
END WHILE

5. Test Functions and Parameter Setting

In order to evaluative the performance of the proposed variant and existing variants of DE algorithm, a comprehensive set of 37 -dimensional benchmark functions is used. These benchmark functions are commonly used for multidimensional global optimization problems. These functions are given in Table 2 along with the equation, search space, dimension, and optimum value of each function. -dimensional functions have been used for extensive comparison of DE algorithm variants for various dimensions.

The control parameters of the DE algorithm are (population size), (dimension), (scaling factor), and CR (crossover control parameter). Different researchers have used different and CR parameter values for function optimization of -dimensional functions. Like Noman and Iba [52] have used , , Brest et al. [10] have used , ; Ali and Torn [53] have used , ; Piotrowski and Napiorkowski [54] have used and ; Dong et al. [55] have used and in their research work. In this research work, we have used control parameters , and population size . The dimensions size used is 10, 20, and 30 and the numbers of training iterations for average fitness value are 5000, 10000, and 15000 for 10D, 20D, and 30D, respectively. The DE algorithm iterates over 30 independent runs and the average fitness value of these 30 runs is used in the results. Same parameter values are used for all functions and all mutation strategies. Experimental results of number of function calls (NFC) are generated for maximum NFC 10⁴DIM [56]. To find out the number of function calls, VTR is set to 0.0001 and Max-NFC for 10D, 20D, and 30D is, respectively, 100,000, 200,000, and 300,000 for all functions and all mutation strategies.

6. Experimental Results and Discussion

This section presents simulation results and its comparative analysis for DE mutation strategies given in Table 1 using parameter setting given in Section 5. In order to perform a fair comparison, all results are averaged over 30 independent runs. Both binomial and exponential crossover schemes are implemented for all benchmark functions given in Table 2 and DE mutation strategies given in Table 1 to obtain the results on a set of benchmark functions given in Table 2 for the DE mutation strategies given in Table 1. Both average fitness value and number of function calls (NFC) are used for performance evaluation of DE mutation strategies as used by most of the evolutionary computing algorithms [14]. First, average fitness values of the functions are considered over 30 independent runs for the performances evaluation of DE mutation strategies. For easy observation, best values of mutation strategies are given as bold face for every case. Experimental results of average fitness for all DE mutation strategies and benchmark functions are reported in Table 2. Because of space limitation, the results of binomial and exponential strategies of proposed TSDE, five most commonly used strategies [14], and one other best performing strategy are reported in Tables 3 and 4. Figures 2–9 show average fitness and NFC Dendrograms for the proposed DE mutation variant and existing DE mutation strategies. Figure 10 shows the convergence graph of binomial and exponential versions of the proposed TSDE and other selected mutation strategies. NFC experimental results for DE mutation strategies and benchmark functions are reported in Table 4.

The results of average fitness value and NFC are used to evaluate the performance of DE mutation strategies. Commonly used mutation strategies, proposed mutation variant and, one other better performing mutation strategy results are included in this section since results are large enough to manage in the tabular form for all variants in the paper. Experimental results are obtained for 40 DE mutation strategies given in Table 1 and the proposed TSDE mutation variant ( and ) but reported for selected mutation strategies only due to space issue. Experimental results are analyzed on the basis of commonly used performance parameters average fitness and NFC. The benchmark functions are -dimensional functions having varying nature. From average fitness results, it can be observed that proposed binomial TSDE() version has dominating performance among all strategies; the mutation strategy that has never announced to be one of the best performing mutation strategies in DE has the second best performance; proposed exponential TSDE() and two other well known DE mutation strategies and have subsequent best performance. Although in most of the cases proposed shares its best performance with other mutation strategies like , , , and so forth, convergence graphs show that has better convergence speed than these mutation strategies. Research result shows that proposed mutation strategies perform better for the functions having various characteristics like unimodal/multimodal and separable/nonseparable. The proposed mutation strategy () has better average fitness performance for separable functions , , , , (10D, 20D), (10D), (10D, 20D), , , , , (10D), , and ; nonseparable functions , , , , , , and ; unimodal functions , , , , , (10D), , and ; and multimodal functions , , , , (10D, 20D), (10D, 20D), , , (10D), , , , and . One of the rarely used DE mutation strategies has second best average fitness performance for separable functions , , (10D), , , (10D), , , , , and ; nonseparable functions , , and (10D, 30D); unimodal functions , (10D), , (10D), , and ; and multimodal functions , , , , (10D, 30D), , , and . The proposed exponential mutation strategy () has better average fitness performance for separable functions (10D, 30D), , , , (10D), and ; nonseparable functions (30D), , , and ; unimodal functions and ; and multimodal functions (10D, 30D), (30D), , (10D), , , , and . Average fitness result shows that has dominating performance among all strategies and although it shares its best performance with other mutation strategies like , , and in most of the cases, the convergence graphs show that the proposed TSDE has fast convergence than other mutation strategies like , , and and vice versa.

Results of number of function calls show that TSDE() has clearly leading performance among all other DE mutation strategies in most of the cases. The binomial TSDE mutation variant has best performance in most of the functions having various characteristics like separable, nonseparable, unimodal, and multimodal. outperforms other strategies for Sphere model, Axis parallel hyperellipsoid, Schwefel’s problem 1.2, Rosenbrock’s valley, Griewank’s function (20D, 30D), Sum of different power, Levy function (30D), Zakharov function, Schwefel’s problem 2.22, Step function, De Jong’s function-4, Levy and Montalvo Problem, Cosine Mixture, Cigar, Function “15,” Ellipse Function, Tablet Function, Schwefel, MultiModal global optimization problem, Quintic global optimization problem, and Stochastic global optimization problem. The exponential version of TSDE mutation variant has better performance for Ackley’s path function (30D), Levy function (20D), and Alpine function (20D). One of the commonly used mutation strategies of DE algorithm has better performance for Griewank’s function (10D), Mishra-1 global optimization problem, Mishra-2 global optimization problem, Stretched-V global optimization problem (30D), and XinSheYang (30D) functions. The mutation strategies that fail to reach VTR for any function with specified parameter contain dashed values in Table 4 for the NFC results. It can be noted from the results that the proposed TSDE performs better than all others including . Proposed TSDE can be one of the powerful mutation strategies in DE algorithm.

Experimental result can be summarized that the proposed binomial “TSDE()” has the dominating performance among all DE mutation strategies for both average fitness values and number of function call values. The strategy (“DE/rand repeated to best/1/bin”) is the second best performing mutation strategy in average fitness value and is runner-up in most of the cases of number of function call values. One interesting aspect of this variant is that the performance of this strategy is never announced to be one the best strategies of the DE algorithm. The mutation strategies and (“DE/best/1/bin”) that are already known to be best performing DE variant have subsequent best performance along with exponential “TSDE()” and vice versa. It can also be summarized that the worst performing mutation strategies among the considered strategies are (“DE/rand repeating & current to rand/1/bin”), , (DE/rand to best/1), (“DE/rand to best/2”) (“DE/rand to current/2”), (“DE/rand/3”), and (“DE/best/3”) and vice versa.

7. Statistical Analysis

The results of DE mutation strategies are analyzed statistically by calculating the rank correlation and by generated Dendrograms. To find the rank correlation, rank of each DE mutation strategy is calculated to find the correlation between DE strategies. The performance of mutation strategies will be observed during the analysis for various dimensions with the help of Dendrogram. Similarity/dissimilarity matrix is used to generate the Dendrogram. Similarity matrixes are obtained using “Statistical Package for the Social Sciences” (SPSS). After similarity matrix, pairwise distances between DE strategies are calculated. Pairwise distance is used to create hierarchical cluster tree linkage that is used to create the Dendrogram. Dendrogram actually generates the clusters from the dissimilarity matrix using agglomerative clustering. Dendrograms of DE mutation strategy for test suit of benchmark functions are shown in Figures 1–3 and Algorithm 1 for different dimensions.

Pairwise dissimilarities between objects can be easily depicted using Dendrograms. Dendrograms are considered to be an effective way of cluster analysis that specifies the level of similarity joining of any two clusters. The leaf of clusters in Dendrograms contains the DE mutation strategy’s number and -axis contains the dissimilarity scale that shows the joining distance between clusters. The height of each branch in Dendrograms shows that which DE strategies are similar or different from each other. The greater height of DE strategies branches means that strategies are more different from each other. Consider the Dendrograms generated from average fitness values and number of function call values given in Figures 2–9, respectively. A similarity measure among the DE strategies is explored by generating Dendrograms from the average fitness values and number of function call performance parameters. The best performing mutation strategies are contained in the similar performing cluster shown in transparent circles and most worst performing strategies are contained in the dotted circle in each Dendrogram. The Dendrograms are generated for 10D, 20D, 30D and the average of all (10D, 20D, and 30D) is generated for both performance metrics of fitness value and number of function calls. The best performing and worst performing strategies among all mutation strategies are contained in the corresponding groups in most of the cases. The best performing strategies “TSDE(),” “DE/rand repeated to best/1/bin (),” “DE/best/1/bin (),” and “()” are contained in the same group in most of the cases that can be considered as best performing mutation strategies that are group shown in the transparent circle in Figures 2–9. The worst performing group of mutation strategies is encircled dashed line and it contains “DE/rand repeating & current to rand/1/bin (),” “DE/rand & current to best/2/bin (),” “DE/rand & current to rand/1/bin (),” “DE/rand/3/bin (),” and “DE/best/3 ()” and vice versa.

It is important to discuss that a number of DE mutation strategies are considered in this research work and all these strategies perform different from each other. Dendrograms are also considered to be an effective tool to determine the clusters number. For DE strategies, more number of clusters in the Dendrogram indicates that DE mutation strategies have varying performance from one another, which shows the diverse nature of these strategies in DE algorithm. The diverse nature of DE variants will be helpful to make this algorithm more broadly applicable in various optimization problems of distinct characteristics.

The convergence graphs of selected benchmark functions for 10 dimensions (10D) are given in Figure 10. Logarithmic convergence graphs of average fitness show number of iterations horizontally and performance vertically. Horizontal iterations are shown as a multiple of 100. Figure 10 contains the convergence graphs of five commonly used DE mutation strategies (, , , , and ), the proposed mutation variants ( and ), and one other better performing variant . Convergence graphs of few selected functions are given in the paper. Convergence graph shows that the proposed “TSDE()” mutation variant has better performance in most of the cases and proposed that “TSDE()” has better performance in few cases. The mutation strategy (“DE/rand to best/1/bin”) proves itself to be another better performing DE strategy; it is following “TSDE ()” in most cases of convergence graphs. It is important to mention here that (“DE/rand to best/1/bin”) strategy has never been declared to be one of the best performing variants or among top performing variants of DE algorithm. The convergence graph clearly shows the dominating performance of the proposed DE mutation variant “TSDE().” Most of the researchers have concentrated on very few basic DE strategies like “DE/rand/1/bin (),” “DE/best/1/bin (),” “DE/rand/2/bin (),” “DE/best/2/bin (),” and so forth, and some key performing mutation strategies remained unnoticed in the DE research. This study will prove to be fruitful for DE researchers having choice of different variants for scientific and real time applications.

8. Comparison of TSDE with PSO and GA

PSO and GA are two well known metaheuristics that are applied on variety of real world problems [57]. The simulation results of proposed “TSDE” is compared with two other well known metaheuristics: particle swarm optimization and genetic algorithm. In order to compare TSDE with PSO and GA, the parameter setting for GA is crossover rate and mutation rate [58] while for PSO inertia weight , , and are used [59].

For fair comparison between PSO, GA, and TSDE, the population size, dimensions, number of iterations, and standard benchmark functions are kept the same. Results of average fitness value and NFC are presented in Tables 5 and 6, respectively. Proposed strategy TSDE has dominating performance than evolutionary metaheuristics PSO and GA in most of the cases of 10D, 20D, and 30D for average fitness value and NFC. TSDE proves itself to be one of the powerful variants of DE algorithm that can be a significant addition in DE research.

9. Conclusion

In this paper, a novel technique TSDE mutation variant based on the selection of parent vectors to generate the new population is proposed. The proposed variants “TSDE()” and “TSDE()” are compared with the existing DE mutation strategies. TSDE() mutation variant has dominating performance among the DE mutation strategies. This new mutation variant TSDE will prove to be a valuable addition in DE literature. A comprehensive set of well known -dimensional benchmark functions have been used to evaluative the performance of the proposed TSDE as well as existing mutation strategies of DE algorithm. In literature, such an extensive comparison performed with many number of mutation strategies is lacking. This comparison unveils some very beneficial facts related to the DE research improvements. The other important aspect of this research is the statistical comparison of the DE mutation strategies; such a comparison is also not available in the literature. Experimental results consist of average fitness value and number of function call parameters of DE algorithm that are reported in Tables 3 and 4. Since total strategies are 42 and it is not possible to report all the mutation strategies in few tables, so proposed TSDE mutation variant, five commonly used mutation strategies, and one better performing mutation strategy are reported in the results. It can be concluded that there is a deviation in the performance of DE algorithm variants and the selection of DE variant affects the performance result of DE algorithm. Better results can be obtained by choosing the better performing strategies. The proposed binomial “TSDE()” has the leading performance among the DE mutation strategies for both fitness value and number of function call performance parameters, the proposed exponential “TSDE(),” mutation strategy “DE/rand to best/1/bin (),” “” and “DE/best/1/bin ()” are the other better performance mutation strategies. This research work also reveals one of the best performing mutation strategies “DE/rand to best/1/bin ()” that has been rarely brought into play. This research work will prove to be a significant addition in DE literature. The results of TSDE are also compared with two well known heuristics, GA and PSO, that show dominating performance of TSDE. The further studies are still required to explore the limitations, advantages, and flaws in this direction. The main focus in this work is not the overwhelming and existing DE variants but new tracks to work on the DE variants in optimization. One possible direction of future work of this paper is to develop a mathematical model for the DE mutation strategies.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

References

R. Storn and K. Price, “Differential evolution—a simple and efficient adaptive scheme for global optimization over continuous spaces,” Tech. Rep. TR-95-012, ICSI, Berkeley, Calif, USA, 1995.
View at: Google Scholar
K. Price, R. M. Storn, and J. A. Lampinen, Differential Evolution: A Practical Approach to Global Optimization, Natural Computing Series, Springer, New York, NY, USA, 1st edition, 2005.
Y. Lu, J. Zhou, H. Qin, Y. Li, and Y. Zhang, “An adaptive hybrid differential evolution algorithm for dynamic economic dispatch with valve-point effects,” Expert Systems with Applications, vol. 37, no. 7, pp. 4842–4849, 2010.
View at: Publisher Site | Google Scholar
I. L. Lopez Cruz, L. G. van Willigenburg, and G. van Straten, “Efficient differential evolution algorithms for multimodal optimal control problems,” Applied Soft Computing Journal, vol. 3, no. 2, pp. 97–122, 2003.
View at: Publisher Site | Google Scholar
A. Qing, “Dynamic differential evolution strategy and applications in electromagnetic inverse scattering problems,” IEEE Transactions on Geoscience and Remote Sensing, vol. 44, no. 1, pp. 116–125, 2006.
View at: Publisher Site | Google Scholar
R. Burman, S. Paul, and S. Das, “A differential evolution approach to multi-level image thresholding using type II fuzzy sets,” in Swarm, Evolutionary, and Memetic Computing, vol. 8297 of Lecture Notes in Computer Science, pp. 274–285, Springer, 2013.
View at: Publisher Site | Google Scholar
R. Xu, G. K. Venayagamoorthy, and D. C. Wunsch II, “Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization,” Neural Networks, vol. 20, no. 8, pp. 917–927, 2007.
View at: Publisher Site | Google Scholar
S.-T. Pan, B.-Y. Tsai, and C.-S. Yang, “Differential evolution algorithm on robust IIR filter design and implementation,” in Proceedings of the 8th International Conference on Intelligent Systems Design and Applications (ISDA '08), pp. 537–542, Kaohsiung, Taiwan, November 2008.
View at: Publisher Site | Google Scholar
R. Angira and B. V. Babu, “Optimization of process synthesis and design problems: a modified differential evolution approach,” Chemical Engineering Science, vol. 61, no. 14, pp. 4707–4721, 2006.
View at: Publisher Site | Google Scholar
J. Brest, S. Greiner, B. Bošković, M. Mernik, and V. Zumer, “Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems,” IEEE Transactions on Evolutionary Computation, vol. 10, no. 6, pp. 646–657, 2006.
View at: Publisher Site | Google Scholar
A. K. Qin, V. L. Huang, and P. N. Suganthan, “Differential evolution algorithm with strategy adaptation for global numerical optimization,” IEEE Transactions on Evolutionary Computation, vol. 13, no. 2, pp. 398–417, 2009.
View at: Publisher Site | Google Scholar
J. Zhang and A. C. Sanderson, “JADE: adaptive differential evolution with optional external archive,” IEEE Transactions on Evolutionary Computation, vol. 13, no. 5, pp. 945–958, 2009.
View at: Publisher Site | Google Scholar
R. Mallipeddi, P. N. Suganthan, Q. K. Pan, and M. F. Tasgetiren, “Differential evolution algorithm with ensemble of parameters and mutation strategies,” Applied Soft Computing Journal, vol. 11, no. 2, pp. 1679–1696, 2011.
View at: Publisher Site | Google Scholar
S. M. Islam, S. Das, S. Ghosh, S. Roy, and P. N. Suganthan, “An adaptive differential evolution algorithm with novel mutation and crossover strategies for global numerical optimization,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 42, no. 2, pp. 482–500, 2012.
View at: Publisher Site | Google Scholar
K. Zielinski, M. Joost, R. Laur, and B. Orlik, “Choosing suitable variants of differential evolution and particle swarm optimization for the optimization of a PI cascade control,” in Proceedings of the 11th International Conference on Optimization of Electrical and Electronic Equipment (OPTIM '08), pp. 55–62, May 2008.
View at: Publisher Site | Google Scholar
V. L. Huang, A. K. Qin, and P. N. Suganthan, “Self-adaptive differential evolution algorithm for constrained real-parameter optimization,” in Proceedings of the IEEE Congress on Evolutionary Computation (CEC '06), pp. 17–24, Vancouver, Canada, July 2006.
View at: Google Scholar
R. Storn, “On the usage of differential evolution for function optimization,” in Proceedings of the Biennial Conference of the North American Fuzzy Information Processing Society (NAFIPS '96), pp. 519–523, Berkeley, Calif, USA, 1996.
View at: Google Scholar
Y. Ao and H. Chi, “Dynamic differential evolution for constrained real-parameter optimization,” Journal of Advances in Information Technology, vol. 1, no. 1, pp. 43–51, 2010.
View at: Publisher Site | Google Scholar
G. T. S. De Oliveira and S. F. P. Saramago, “A contribution to the study about differential evolution,” Ciência & Engenharia, vol. 16, pp. 1–8, 2007.
View at: Google Scholar
T. Podoba, L. Tomsu, K. Vlcek, and M. Heczko, “Surface reconstrucion by means of AI,” in Proceedings of the International Conference on Computational Experimental Engineering and Sciences, pp. 111–121, 2010.
View at: Google Scholar
I. Zelinka, R. Senkerik, M. Bialic-Davendra, and D. Davendra, “Chaos driven evolutionary algorithm for the traveling salesman problem,” in Traveling Salesman Problem, Theory and Applications, D. Davendra, Ed., chapter 4, pp. 55–70, InTech, Rijeka, Croatia, 2010.
View at: Publisher Site | Google Scholar
M. Ali, M. Pant, and A. Abraham, “Simplex differential evolution,” Acta Polytechnica Hungarica, vol. 6, no. 5, pp. 95–115, 2009.
View at: Google Scholar
S. Almeida-Luz, M. A. Vega-Rodŕiguez, J. A. Gómez-Pulido, and J. M. Śanchez-Perez, “Applying differential evolution to the reporting cells problem,” in Proceedings of the International Multiconference on Computer Science and Information Technology (IMCSIT '08), pp. 65–71, IEEE, Wisła, Poland, October 2008.
View at: Publisher Site | Google Scholar
G. Jeyakumar and C. S. Velayutham, “An empirical performance analysis of differential evolution variants on unconstrained global optimization problems,” International Journal of Computer Information Systems and Industrial Management Applications, vol. 2, pp. 77–86, 2010.
View at: Google Scholar
I. Triguero, S. García, and F. Herrera, “Differential evolution for optimizing the positioning of prototypes in nearest neighbor classification,” Pattern Recognition, vol. 44, no. 4, pp. 901–916, 2011.
View at: Publisher Site | Google Scholar
M. Weber, Parallel global optimization: structuring populations in differential evolution [Ph.D. thesis], University of Jyväskylä, Jyväskylä, Finland, 2010.
S. M. Elsayed, R. A. Sarker, and D. L. Essam, “Differential evolution with multiple strategies for solving CEC2011 real-world numerical optimization problems,” in Proceedings of the IEEE Congress of Evolutionary Computation (CEC '11), pp. 1041–1048, June 2011.
View at: Publisher Site | Google Scholar
B. Xin, J. Chen, J. Zhang, H. Fang, and Z.-H. Peng, “Hybridizing differential evolution and particle swarm optimization to design powerful optimizers: a review and taxonomy,” IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 42, no. 5, pp. 744–767, 2012.
View at: Publisher Site | Google Scholar
B. Y. Qu and P. N. Suganthan, “Novel multimodal problems and differential evolution with ensemble of restricted tournament selection,” in Proceedings of the IEEE World Congress on Computational Intelligence, pp. 1–7, Barcelona, Spain, 2010.
View at: Google Scholar
M. G. Epitropakis, V. P. Plagianakos, and M. N. Vrahatis, “Finding multiple global optima exploiting differential evolution’s niching capability,” in Proceedings of the IEEE Symposium on Differential Evolution (SDE '11), pp. 1–8, IEEE, Paris, France, April 2011.
View at: Publisher Site | Google Scholar
C. Qiu, M. Liu, and W. Gong, “Differential evolution with tournament-based mutation operators,” International Journal of Computer Science Issues, vol. 10, no. 2, pp. 180–187, 2013.
View at: Google Scholar
S. Das, A. Abraham, U. K. Chakraborty, and A. Konar, “Differential evolution using a neighborhood-based mutation operator,” IEEE Transactions on Evolutionary Computation, vol. 13, no. 3, pp. 526–553, 2009.
View at: Publisher Site | Google Scholar
A. M. Sutton, M. Lunacek, and L. D. Whitley, “Differential evolution and non-separability: using selective pressure to focus search,” in Proceedings of the 9th Annual Genetic and Evolutionary Computation Conference, pp. 1428–1435, London, UK, July 2007.
View at: Publisher Site | Google Scholar
M. G. Epitropakis, D. K. Tasoulis, N. G. Pavlidis, V. P. Plagianakos, and M. N. Vrahatis, “Enhancing differential evolution utilizing proximity-based mutation operators,” IEEE Transactions on Evolutionary Computation, vol. 15, no. 1, pp. 99–119, 2011.
View at: Publisher Site | Google Scholar
D. Zaharie, “A comparative analysis of crossover variants in differential evolution,” in Proceedings of the International Multiconference on Computer Science and Information Technology, pp. 171–181, 2008.
View at: Google Scholar
B. V. Babu and S. A. Munawar, “Differential evolution strategies for optimal design of shell-and-tube heat exchangers,” Chemical Engineering Science, vol. 62, no. 14, pp. 3720–3739, 2007.
View at: Publisher Site | Google Scholar
E. Mezura-Montes, J. Velázquez-Reyes, and C. A. Coello Coello, “A comparative study of differential evolution variants for global optimization,” in Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation (GECCO '06), pp. 485–492, Washington, DC, USA, 2006.
View at: Publisher Site | Google Scholar
M. Ali, M. Pant, A. Abraham, and V. Snasel, “Differential evolution using mixed strategies in competitive environment,” International Journal of Innovative Computing, Information and Control, vol. 7, no. 8, pp. 5063–5084, 2011.
View at: Google Scholar
Y. Zhang, S. Wang, P. Phillips, and G. Ji, “Binary PSO with mutation operator for feature selection using decision tree applied to spam detection,” Knowledge-Based Systems, vol. 64, pp. 22–31, 2014.
View at: Publisher Site | Google Scholar
S. M. Islam, S. Das, S. Ghosh, S. Roy, and P. N. Suganthan, “An adaptive differential evolution algorithm with novel mutation and crossover strategies for global numerical optimization,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 42, no. 2, pp. 482–500, 2012.
View at: Publisher Site | Google Scholar
Y. Wang, Z. Cai, and Q. Zhang, “Differential evolution with composite trial vector generation strategies and control parameters,” IEEE Transactions on Evolutionary Computation, vol. 15, no. 1, pp. 55–66, 2011.
View at: Publisher Site | Google Scholar
W. Gong, Z. Cai, C. X. Ling, and H. Li, “Enhanced differential evolution with adaptive strategies for numerical optimization,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 41, no. 2, pp. 397–413, 2011.
View at: Publisher Site | Google Scholar
S. Das, A. Abraham, U. K. Chakraborty, and A. Konar, “Differential evolution using a neighborhood-based mutation operator,” IEEE Transactions on Evolutionary Computation, vol. 13, no. 3, pp. 526–553, 2009.
View at: Publisher Site | Google Scholar
R. Storn and K. Price, “Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces,” Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.
View at: Publisher Site | Google Scholar | MathSciNet
C.-K. Ting and C.-H. Huang, “Varying number of difference vectors in differential evolution,” in Proceedings of the IEEE Congress on Evolutionary Computation, pp. 1351–1358, Chiayi, Taiwan, May 2009.
View at: Publisher Site | Google Scholar
B. L. Miller and D. E. Goldberg, “Genetic algorithms, tournament selection, and the effects of noise,” Complex Systems, vol. 9, no. 3, pp. 193–212, 1995.
View at: Google Scholar | MathSciNet
T. Blickle and L. Thiele, “A comparison of selection schemes used in genetic algorithms,” TIK-Report 11, TIK Institut fur Technische und Kommunikationsnetze, Swiss Federal Institute of Technology, Zurich, Switzerland, 1995.
View at: Google Scholar
H. Takagi and D. Pallez, “Paired comparisons-based interactive differential evolution,” in Proceedings of the World Congress on Nature & Biologically Inspired Computing, pp. 475–480, Coimbatore, India, 2009.
View at: Google Scholar
P. Pošík and J. Kubalík, “Experimental comparison of six population-based algorithms for continuous black box optimization,” Evolutionary Computation, vol. 20, no. 4, pp. 483–508, 2012.
View at: Publisher Site | Google Scholar
M. G. Epitropakis, V. P. Plagianakos, and M. N. Vrahatis, “Balancing the exploration and exploitation capabilities of the differential evolution algorithm,” in Proceedings of the IEEE World Congress on Computational Intelligence, IEEE Congress on Evolutionary Computation (CEC '08), pp. 2686–2693, Hong Kong, June 2008.
View at: Publisher Site | Google Scholar
R. Mendes and A. S. Mohais, “DynDE: a differential evolution for dynamic optimization problems,” in Proceedings of the IEEE Congress on Evolutionary Computation (IEEE CEC '05), pp. 2808–2815, September 2005.
View at: Google Scholar
N. Noman and H. Iba, “Accelerating differential evolution using an adaptive local search,” IEEE Transactions on Evolutionary Computation, vol. 12, no. 1, pp. 107–125, 2008.
View at: Publisher Site | Google Scholar
M. M. Ali and A. Torn, “Population set-based global optimization algorithms: some modifications and numerical studies,” Computers & Operations Research, vol. 31, no. 10, pp. 1703–1725, 2004.
View at: Publisher Site | Google Scholar | MathSciNet
A. P. Piotrowski and J. J. Napiorkowski, “The grouping differential evolution algorithm for multi-dimensional optimization problems,” Control and Cybernetics, vol. 39, no. 2, pp. 527–550, 2010.
View at: Google Scholar | MathSciNet
X.-L. Dong, S.-Q. Liu, T. Tao, S.-P. Li, and K.-L. Xin, “A comparative study of differential evolution and genetic algorithms for optimizing the design of water distribution systems,” Journal of Zhejiang University: Science A, vol. 13, no. 9, pp. 674–686, 2012.
View at: Publisher Site | Google Scholar
Y. Wang, Z. Cai, and Q. Zhang, “Enhancing the search ability of differential evolution through orthogonal crossover,” Information Sciences, vol. 185, pp. 153–177, 2012.
View at: Publisher Site | Google Scholar | MathSciNet
Y. Zhang, S. Wang, Y. Sun, G. Ji, P. Phillips, and Z. Dong, “Binary structuring elements decomposition based on an improved recursive dilation-union model and RSAPSO method,” Mathematical Problems in Engineering, vol. 2014, Article ID 272496, 12 pages, 2014.
View at: Publisher Site | Google Scholar
Y. Yoon and H. Kim, “The roles of crossover and mutation in real-coded genetic algorithms,” in Bio-Inspired Computational Algorithms and Their Applications, S. Gao, Ed., InTech, 2012.
View at: Google Scholar
R. Poli, J. Kennedy, and T. Blackwell, “Particle swarm optimization,” Swarm Intelligence, vol. 1, no. 1, pp. 33–57, 2007.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2015 Qamar Abbas et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1900

Downloads

806

Citations