Fusion of Computational Intelligence Techniques and Their Practical ApplicationsView this Special Issue
Research Article | Open Access
An Enhanced Differential Evolution with Elite Chaotic Local Search
Differential evolution (DE) is a simple yet efficient evolutionary algorithm for real-world engineering problems. However, its search ability should be further enhanced to obtain better solutions when DE is applied to solve complex optimization problems. This paper presents an enhanced differential evolution with elite chaotic local search (DEECL). In DEECL, it utilizes a chaotic search strategy based on the heuristic information from the elite individuals to promote the exploitation power. Moreover, DEECL employs a simple and effective parameter adaptation mechanism to enhance the robustness. Experiments are conducted on a set of classical test functions. The experimental results show that DEECL is very competitive on the majority of the test functions.
Numerous problems in science and engineering can be converted into optimization problems. Therefore, it is of significance both in theory and in engineering applications to develop effective and efficient optimization algorithms for solving complex problems of science and engineering. Differential evolution (DE), proposed by Storn and Price in 1997 , is a simple yet effective global optimization algorithm. According to frequently reported theoretical and experimental studies, DE has exhibited competitive performance than many other evolutionary algorithms in terms of both convergence speed and solution precision over several benchmark functions and real-life problems [2–4]. Due to its simplicity, easy implementation, and efficiency, DE has stimulated many researchers’ interests since its development. Therefore, it has become a hot research topic in evolutionary computation over the past decades [5–7].
However, its search ability should be further enhanced to obtain better solutions when DE is used to solve various real-life optimization problems [2, 8, 9]. Particularly, DE may suffer from premature convergence and/or slow convergence when solving complex multimodal optimization problems. In order to improve the performance of the conventional DE, a number of DE variants have been proposed in recent decades [2, 6, 10]. Recognizing that the performance of DE depends on the control parameters, Brest et al.  presented a self-adaptive DE (jDE), in which both and are created independently for each individual by an adaptive mechanism. Specifically, the new is created by a random value from 0.1 to 0.9 with a probability 0.1 during the search process. Meanwhile, the new obtains a random value from 0.0 to 1.0 with a probability 0.1. Unlike jDE, JADE, proposed by Zhang and Sanderson , utilizes a distinct parameter adaptation mechanism, in which the new and are created for each individual by a normal distribution and a Cauchy distribution, respectively. In addition, JADE learns knowledge from the recent successful and and applies the learned knowledge for creating new and . Identifying that both the mutation strategies and their associated control parameters can directly influence the performance of DE, Qin et al.  proposed a novel self-adaptive DE, SaDE, which adaptively tunes the trial vector generation strategies and their associated control parameter values by extracting knowledge from the previous search process in generating promising solutions. Mallipeddi et al.  introduced an improved DE with ensemble of parameters and mutation strategies (EPSDE), which employs a pool of diverse trial vector generation strategies and a pool of values for the control parameters and . By incorporating an opposition-based learning strategy into the traditional DE for population initialization and generating new solutions, Rahnamayan et al.  proposed an opposition-based DE (ODE). The experimental results confirmed that the opposition-based learning strategy can improve the convergence speed and the solution accuracy of DE. Further, Wang et al.  improved the opposition-based learning strategy, proposed a generalized opposition-based learning strategy, and presented an enhanced DE with generalized opposition-based learning strategy (GODE). Jia et al.  presented an effective memetic DE algorithm, DECLS, which utilizes a chaotic local search with a shrinking strategy to improve the search ability. Experimental results indicated that the performance of the canonical DE is significantly improved by the chaotic local search. Recently, Wang et al.  proposed a composite DE, called CoDE, the main idea of which is to randomly combine several well studied trial vector generation strategies with a number of control parameter settings highly recommended by other researchers at each generation to create new trial vectors. Experimental results on all the CEC2005 contest test instances show that CoDE is very competitive.
Although there already exist many DE variants for solving complex optimization problems, according to the no free lunch (NFL) theory , the performance of DE for some benchmark functions and real-life problems should be further enhanced to obtain better solutions. Moreover, many studies have revealed that embedding local search strategy can greatly enhance the search ability of DE [14, 16, 19]. Motivated by these considerations, in order to promote the performance of DE on complex optimization problems, this study proposes an enhanced differential evolution with elite chaotic local search, called DEECL. In DEECL, we utilize a chaotic search strategy based on the heuristic information from the elite individuals to promote the exploitation power. Further, we also design a simple and effective parameter adaptation mechanism to enhance the robustness.
The rest of the paper is organized as follows. The conventional DE is introduced in Section 2. Section 3 presents the enhanced DE. Numerical experiments are presented in Section 4 for the comparison and analysis. Finally, the paper is concluded in Section 5.
2. Differential Evolution
Without loss of generality, only minimization problems are considered in this study. We suppose that the objective function to be minimized is Min , , and the search space is where is the number of dimensions of the problem, and denote the lower and upper boundaries of the search space, respectively.
Similar to other evolutionary algorithms, DE also has a simple structure, only including three simple operators, namely, mutation, crossover, and selection operators . In the initial phase, DE creates an initial population , which is randomly generated from the search space, where , ; is the population size and is the generation. After initialization, the mutation and crossover operators are performed to create the trial vectors, and then the selection operator is utilized to select the better one between the offspring individual and the parent individual for the next generation. DE performs these steps repeatedly to converge toward the global optima until the terminating criterion is reached . In the following subsections, the evolutionary operators of DE will be introduced in detail.
2.1. Mutation Operator
In the mutation operator, a mutant vector is created by using a predetermined mutation strategy for each individual , namely, target vector, in the current population . DE has many mutation strategies used in its implementations, such as DE/rand/1, DE/best/1, DE/rand-to-best/1, DE/best/2 and DE/rand/2 . Among these mutation strategies, DE/rand/1 is the most frequently used mutation strategy, which is expressed as follows :where , , and are randomly selected from the set , and they are mutually different from each other. is called as scaling factor, amplifying the difference vector .
2.2. Crossover Operator
Following mutation, a trial vector is generated by executing the crossover operator for each pair of target vector and its corresponding mutant vector . Binomial crossover is the most commonly used crossover operators in current popular DE. The binomial crossover is described as follows : where is generated for each and takes a value from 0.0 to 1.0 in a uniformly random manner, and is the crossover probability, which limits the number of parameters inherited from the mutant vector . The integer is randomly chosen from the range , which guarantees that at least one parameter of the trial vector is inherited from the mutant vector .
2.3. Selection Operator
Like the genetic algorithm, the selection process of DE is also based on the Darwinian law of survival of the fittest. The selection process is performed in order to choose the more excellent individuals for the next generation. For minimization problems, the selection operator can be defined in the following form : where and indicate the fitness values of the target vector and its corresponding trial vector , respectively.
2.4. Algorithmic Framework of DE
Based on the above elaborate introduction of the DE’s operators, we present the framework of DE with DE/rand/1/bin strategy in Algorithm 1, where is the number of fitness evaluations, is the maximum number of evaluations, rand(0,1) indicates a random real number in the range , represents a random integer in the range , and is the global best individual found so far.
3. Proposed Approach
DE has been demonstrated to yield superior performance for solving various real-world optimization problems [21–23]. However, it tends to suffer from premature convergence and/or slow convergence when solving complex optimization problems [6, 24]. To enhance the performance of DE, many researchers have proposed various improved DE algorithms during the past decade [25–27]. Among the DE variations, memetic method is a promising approach to improve the performance of the traditional DE, which utilizes various local search strategies, such as chaotic search strategy , simplex crossover search strategy , and orthogonal search strategy , to strengthen the exploitation ability of the traditional DE and consequently accelerate the convergence speed. Among the local search strategies commonly used in memetic DE, chaotic search strategy is inspired by the chaos phenomenon in nature. Chaos is a classic nonlinear dynamical system, which is widely known as a system with the properties of ergodicity, randomicity, and sensitivity to its initial conditions [16, 29, 30]. Due to its ergodicity and randomicity, a chaotic system can randomly generate a long-time sequence which is able to traverse through every state of the system and every state is generated only once if given a long enough time period [16, 31]. Taking advantage of the well-known characteristics of the chaotic systems, researchers have proposed many chaotic search strategies for optimizing various problems [16, 32–34]. However, to the best of our knowledge, among many chaotic search strategies, they pay more attention to the characteristics of the ergodicity and randomicity of the chaotic system. Therefore, the exploration capacity can be indeed improved. However, in order to maintain a balance between exploration and exploitation, the exploitation ability of the chaotic search strategy should be further enhanced. Thus, when designing a relatively comprehensive chaotic search strategy, we should further integrate more heuristic information into the chaotic search strategy to promote its exploitation power. Generally, the elite individuals in the current population known as a promising search direction toward the optimum are the favorable source that can be employed to enhance the exploitation ability. Based on these considerations, we present an elite chaotic search strategy, which not only utilizes the characteristics of the ergodicity and randomicity of the chaotic system, but also merges the superior information of the current population into the chaotic search process.
3.2. Elite Chaotic Search
In many chaotic search strategies, the Logistic chaotic function is utilized to generate a chaotic sequence, which is formulated as follows : where is the initial value of the chaotic system, which is randomly generated from the range , but cannot be equal to 0.25, 0.5, or 0.75. is the th state of the chaotic system. As known, the initial state of the chaotic system is randomly produced. Due to its ergodicity and sensitivity to the initial state , is a random long-time sequence, which can traverse through every state of the system and every state is generated only once if is large enough.
In order to enhance the exploitation ability of the traditional chaotic search strategy, we integrate the heuristic information learned from the elite individuals into the chaotic search strategy to promote the exploitation power. The proposed elite chaotic search strategy is defined by where is an individual to be performed the elite chaotic search, which is randomly chosen from the current population. is the chaotic sequence, where , , and is an elite individual, which is randomly chosen from the top individuals in the current population with .
In the proposed elite chaotic search operator, an individual is randomly selected from the current population to undergo the elite chaotic search strategy. After that, the initial value of the chaotic system takes a value from range in a uniformly random manner. Then, an elite chaotic search procedure for individual is repeatedly performed until finding a better solution than individual or the number of iterations is equal to . The framework of the elite chaotic search operator is described in Algorithm 2.
3.3. Parameter Adaptation
Since the setting of control parameters can significantly influence the performance of DE, parameter adaptation mechanism is essential for an efficient DE [7, 11, 12]. To this end, we design a simple and effective parameter adaptation mechanism inspired by  into DEECL. In DEECL, each individual is independently associated with its own mutation factor and crossover probability . For individual , its control parameters and are initialized to 0.5 and 0.9, respectively. Generally, a normal distribution with mean value 0.5 and standard deviation 0.3 is a promising adaptive approach for the mutation factor of DE , whereas Cauchy distribution is more favorable to diversify the mutation factors and thus avoid premature convergence . Based on these considerations, at each generation, the new mutation factor associated with individual is generated by a Cauchy distribution random real number with location parameter 0.5 and scale parameter 0.3 with probability 0.1. Additionally, following the suggestions in , the new crossover probability associated with individual acquires a random value from 0.0 to 1.0 with probability 0.1. Mathematically, the new control parameters and associated with individual for generating its corresponding trial vector are obtained by where is a Cauchy distribution random real number with location parameter 0.5 and scale parameter 0.3 and is a uniformly random number within the range . After obtaining the new control parameters and , the corresponding trial vector are created by using the new control parameters and . It is widely acknowledged that better control parameter values tend to produce better individuals that have a greater chance to survive and thus these values should be propagated to the next generations . Therefore, in the selection step, the control parameters and associated with individual for the next generation are updated by From the above designed parameter adaptation mechanism, we can infer that the better control parameters of DEECL can be propagated to the next generations. Therefore, the control parameters of DEECL can be adaptively tuned according to the feedback from the search process.
4. Numerical Experiments
4.1. Experimental Setup
In order to assess the performance of the proposed DEECL, we use 13 classical test functions () that are widely used in the evolutionary computation community [8, 12, 35] to verify the effectiveness of the proposed DEECL. We describe these test functions in Table 1. Among these test functions , are continuous unimodal functions. is the Rosenbrock function which is unimodal for and 3; however, it may have multiple minima in high dimension cases . is a discontinuous step function, and is a noisy function. are multimodal functions and they exist many local minima .
In all experiments, we set the number of dimensions to 30 for all these test functions. We carry out 30 independent runs for each algorithm and each test function with 150,000 function evaluations (FES) as the termination criterion. Moreover, we record the average and standard deviation of the function error value for estimating the performance of the algorithms, as recommended by , where is the best solution gained by the algorithm in a run and is the global optimum of the test function.
4.2. Benefit of the Two Components
There are two important components in the proposed DEECL: the proposed elite chaotic search strategy and the designed parameter adaptation mechanism. Accordingly, it is interesting to recognize the benefit of the two components of the proposed DEECL. For this purpose, we conduct experiments to compare the proposed DEECL with the traditional DE with DE/rand/1 strategy and two variants of DEECL, namely, DE with the proposed elite chaotic search strategy (DEwEC) and DE with the designed parameter adaptation mechanism (DEwPA). In the experiments, we set the population size of all the algorithms to 100. For the other parameters of DE and DEwEC, we set and , following the suggestions in .
We present the experimental results of the above mentioned algorithms in Table 2. The best results among the four algorithms are highlighted in boldface. “Mean Error” and “Std Dev” indicate the mean and standard deviation of the function error values achieved in 30 independent runs, respectively. From the results of comparison between DE and DEwEC, DEwEC performs better than DE on all test functions with the exception of . On test function , both DE and DEwEC exhibit similar performance. In total, DEwEC is better than DE on twelve test functions. The results of comparison between DE and DEwEC indicate that our introduced elite chaotic search strategy is effective to enhance the performance of the traditional DE.
From the comparison of DE with DEwPA, DEwPA surpasses DE on all test functions except for and . On test function , both DE and DEwPA demonstrate similar performance, whereas DE is better than DEwPA on test function . In summary, DEwPA outperforms DE on eleven test functions. The comparison of DE with DEwPA reveals that our designed parameter adaptation mechanism is capable of improving the efficiency of the traditional DE.
By incorporation of both the proposed elite chaotic search strategy and the designed parameter adaptation mechanism, DEECL achieves promising performance, which is better than other three DE algorithms on the majority of the test functions. To be specific, DEECL is better than DE, DEwEC, and DEwPA on eleven, nine, and ten test functions, respectively. DE, DEwEC, and DEwPA can outperform DEECL only on one test function. Comparison results suggest that both the introduced elite chaotic search strategy and the designed parameter adaptation mechanism demonstrate positive effect on the performance of DEECL. In addition, the comparison results confirm that the introduced elite chaotic search strategy and the designed parameter adaptation mechanism can help DE with both outperform DE with either or neither one on the majority of the test functions. Moreover, the introduced elite chaotic search strategy and the designed parameter adaptation mechanism work together to improve the performance of the traditional DE rather than contradict each other. The evolution of the average function error values derived from DE, DEwEC, DEwPA, and DEECL versus the number of FES is plotted in Figure 1 for some typical test functions. As can be seen from Figure 1, DEECL converges faster than DE, DEwEC, and DEwPA.
4.3. Comparison with Other DE Variants
In order to verify the effectiveness of the proposed DEECL algorithm, we compare DEECL with the traditional DE and three other DE variants, namely, jDE , ODE , and DECLS . In addition, jDE is a self-adaptive DE, in which both parameters and are generated independently for each individual by an adaptive mechanism . ODE is proposed by Rahnamayan et al. , which incorporates the opposition-based learning strategy into the traditional DE for population initialization and creating new solutions. DECLS is an effective memetic DE algorithm , which utilizes the chaotic local search strategy and an adaptive parameter control approaches similar to jDE  to improve the search ability. In the experiments, in order to have a fair comparison, we set the population size of all the algorithms to 100. The other parameter settings of these three DE variants are the same as in their original papers.
The mean and standard deviation of the function error values achieved by each algorithm for the 13 classical test functions are presented in Table 3. For convenience of analysis, the best results among the four DE algorithms are highlighted in boldface. In order to gain statistically significant conclusions, we conduct two-tailed -tests at the significance level of 0.05 [28, 35] on the experimental results. The summary comparison results are described in the last three rows of Table 3. “+,” “−,” and “” suggest that DEECL is better than, worse than, and similar to the corresponding algorithm in terms of the two-tailed -tests at the significance level of 0.05, respectively.