Adaptive Cuckoo Search Algorithm for Unconstrained Optimization
Modification of the intensification and diversification approaches in the recently developed cuckoo search algorithm (CSA) is performed. The alteration involves the implementation of adaptive step size adjustment strategy, and thus enabling faster convergence to the global optimal solutions. The feasibility of the proposed algorithm is validated against benchmark optimization functions, where the obtained results demonstrate a marked improvement over the standard CSA, in all the cases.
The solutions to multitudinous domains-whether in engineering design, operational research, industrial process, or economics inevitably have optimization at heart. However, having a solid grasp for such problems turns out to be painstakingly tough and tedious; and thus, gearing towards an efficient and effective algorithm in the light of solving increasingly complex optimization problems in practice is of paramount significance. Extensive and intensive studies in this aspect fruit in numerous optimization techniques, particularly, the bioinspired metaheuristic methods which draw inspiration from the means on how humans and living creatures struggle to survive in a challenging environment, for instance, genetic algorithm (GA) , particle swarm optimization (PSO) , differential evolution (DE) , ant colony optimization , artificial bee colony algorithm , and firefly algorithm , form the hot topics in this area.
Cuckoo search algorithm (CSA), another adoption of biomimicry in the optimization technique which reproduces the breeding strategy of the best known brood parasitic bird, the cuckoos, has been proposed by Yang and Deb recently [7, 8]. Cuckoos, probably one of the most vicious and cunning species of all bird breeds, clandestinely lay their eggs in the nests of other host birds, sparing themselves the parental responsibilities of raising the young. In fact, cuckoos practice the art of deception all the time in their reproductive life. They mimic the colour and pattern of the host eggshell in order to disguise their eggs from being detected by the host birds. To make more space and food for their young chick, cuckoos will steal the host egg while sneaking their own into the nest. However, the relationship between the host species and the cuckoos is often a continuous arms race. The hosts learn to discern the imposters and they either throw out the parasitic eggs or desert the nest; the parasites improve the forgery skill to make their eggs to appear more alike with the host eggs.
The feasibility of applying the CSA to locate the global optimum for the optimization problems has been investigated in the literature. In the pioneering work of Yang and Deb, the CSA has been implemented successfully in optimizing several benchmark functions, and their findings showed that the global search ability of the CSA is more efficient than GA and PSO [7, 8]. On the other hand, the CSA has been employed in diverse domains since its inception; including engineering design process [9–12], chaotic system , wireless sensor networks [14, 15], structural optimization problem [9, 16, 17], image processing [18, 19], milling process , and scheduling problem [21–23]. Undoubtedly, its popularity increases unceasingly in the not-to-distant future.
Nevertheless, in real world situations, obtaining the exact global optimum is impracticable, as the underlying problems are always subjected to various uncertainties and constraints. In this case, instead of finding the actual optimum, the core consideration in selecting an appropriate optimization technique is how much improvement is achievable for a given application at a plausible computational complexity, with an acceptable error. The main thrust of this paper is therefore geared towards a modified CSA, which integrates an accelerated searching strategy in its computation. The improvement over the CSA is tested and validated through the optimization of several benchmarks. The paper is organized as follows. In Section 2, the standard CSA is introduced and its deficiencies are discussed. The modified CSA, specifically, the adaptive cuckoo search algorithm (ASCA), is proposed in Section 3, and the comparative results in evaluating the benchmark optimization functions are presented in Section 4. Finally, some conclusions are drawn in Section 5.
2. Cuckoo Search Algorithm
The CSA, which draws inspiration from cuckoo’s adaption to breeding and reproduction, is idealized with the assumptions as follows:(i)each cuckoo lays one egg in a randomly selected host nest at a time, where the egg represents the possible solution for the problem under study;(ii)the CSA follows the survival of the fittest principle. Only the fittest among all the host nests with high quality eggs will be passed on to the next generation;(iii)the number of host nests in the CSA is fixed beforehand. The host bird spots the intruder egg with a probability . For such incidents, the host bird will either evict the parasitic egg or abandon the nest totally and seek for a new site to rebuild the nest.
Derived from these assumptions, the steps involved in the computation of the standard CSA are presented in Algorithm 1 .
Such idealized assumptions in the CSA, similarly to what was previously proposed in other metaheuristic optimization approaches, make use of the ideas of elitism, intensification, and diversification. Start with a population of possible solutions, a new and potentially better solution (cuckoo egg) is generated. If the fitness value of this cuckoo egg is higher than another randomly selected egg from the available host nests, then it will replace that poor solution.
The cuckoo lays an egg at random location via Lévy flight, which is characterized by where is the possible solution, iter denotes the current generation number, is the gamma function which is defined by the integral: , and is a constant .
The Lévy flight process is a random walk that forms a series of instantaneous jumps chosen from a heavy-tailed probability density function . The step size , which controls the scale of this random search patterns, helps exploit the search space around the current best solution and meanwhile explore the search space more thoroughly by far field randomization such that the search process can effectively move away from a local optima. Therefore, the value for must be assigned judiciously. The search process will be less efficient if α is chosen as a small value, since the location for the new generated solution is near to the previous. On the other hand, if the value for is too big, the new cuckoo egg might be placed outside the bounds. To balance the effectiveness for both intensification and diversification, Yang and Deb assigned the value of as 1 .
Yang and Deb have also pointed out that the CSA outperforms the GA and the PSO in terms of the number of parameters to be adjusted . In the CSA, only the probability of the abandoned nests is tuned. However, the setting of is sufficient enough, as they found out that the convergence rate of CSA is insensitive to . Thus, the fraction of nests to desert is assigned as 0.25 in this study.
3. Adaptive Cuckoo Search Algorithm
For any optimization approach, finding the optimum solutions competently and accurately relies utterly on the inherent search process. The effectiveness of the standard CSA is unquestionable, meaning that when given enough computation time, it is guaranteed to converge to the optimum solutions eventually. However, the search process may be time consuming, due to the associated random walk behavior . In order to improve the convergence rate while maintaining the eye-catching characteristics of the CSA, an accelerated searching process which, similarly to the inertia weight control strategy in the PSO , is proposed here.
The step size , which manages the local and global searching, is assigned as constant in the standard CSA, where is applied. In this present work, a new adaptive cuckoo search algorithm (ACSA) is presented. Instead of using a constant value, the step size is adjusted adaptively in the proposed ACSA, based on the assumption that the cuckoos lay their eggs at the area with a higher egg survival rate. In this regard, by adjusting the step size adaptively, the cuckoos search around the current good solutions for laying an egg as this region probably will contain the optimal solutions, and, on the contrary, they explore more rigorously for a better environment if the current habitat is not suitable for breeding. The step size is determined adaptively as follows: where is the predefined minimum step size, is the predefined maximum step size, is the fitness value of the th cuckoo egg, and and denote the minimum and the average fitness values of all host nests, respectively. The flow of the ACSA is given in Algorithm 2.
The step size determines how far a new cuckoo egg is located from the current host nest. Specifying the minimum and the maximum Lévy flight step size values properly is crucial such that the search process in the ACSA is neither too aggressive nor too ineffective. The and are chosen based on the domain of . Additionally, as precautionary measure, if the ACSA generates a cuckoo egg that falls outside the domain of interest, its position will remain unchanged.
4. Numerical Simulations
To evaluate the feasibility of the proposed ACSA, the algorithm is applied to optimize the five benchmark functions with known global optima, where two of which are unimodal and three of which are multimodal. The optimization performance is compared with the standard CSA. For each test function, the initial populations of 20 host nests are generated randomly. The simulations are performed for 30 independent runs. The optimization process stops if the best fitness value is less than a given tolerance . For both the CSA and ACSA, the Euclidean distance from the known global minimum to the location of the best host nest with the lowest fitness value is evaluated in each iteration. The average of the distance difference for each loop from all the 30 trials is then measured.
In addition, to authenticate the statistical significance of the proposed ACSA, the two-tailed -test is applied. The null hypothesis is rejected at the confidence interval of 5% level, if the difference of the means of both CSA and ACSA is statistically significant. The results are presented in the last column of Table 1.
4.1. Ackley’s Function
The Ackley’s function is a multimodal function which is described as  where and data dimension, , has a global minima of , at . The and are chosen as 0.2domain and 0.8domain , respectively. Figure 1 compares the relative performances of the CSA and ACSA in optimizing Ackley’s function. As observed in this figure, apparently the convergence rate of the ACSA is better than the standard CSA on this test function. Considering Table 1 which summarizes the average cycles needed for both the algorithms to meet the stopping criterion, it can be clearly seen that the CSA takes more iteration to converge; however, the proposed ACSA reaches the global optima about one time faster on average. The ACSA is able to find the global solution, approximately after 2500 fitness function evaluations, as opposed to CSA which reaches the global solution after 4000 objective function evaluations.
4.2. De Jong’s Function
As the simplest unimodal test function, the de Jong’s function is given by  where and . This is a sphere function, with the known global minimum of , at . The and are chosen as 0.2domain and 0.8domain , respectively. The results achieved by the standard CSA and the proposed ACSA in optimizing the de Jong’s function are presented in Figure 2. Due to the simplicity of the test function, it can be found that both algorithms take considerably less iteration steps for convergence but, the ACSA with adjustable step size is much more efficient than the standard CSA. As shown in Table 1, the ACSA reaches the known global optimum in a mean of 1000 cycles, while the CSA requires a longer processing time in order to converge.
4.3. Griewank’s Function
Figure 3 depicts the optimization results of the CSA and ACSA in terms of convergence characteristics for the multimodal Griewank’s function : where with . This is a high dimensional multimodal function, with the known global minimum of at . The and are chosen as 0.02domain and 0.08domain , respectively. Considering Figure 3 and Table 1, the obtained results suggest that the ACSA shows improved convergence efficiency. Moreover, it can be inferred from Table 1 that the superiority of the ACSA is obvious, as it outperforms the CSA in terms of the best, worst, average, and standard deviation, for the number of iterations needed in order to reach the known global optimum.
4.4. Rastrigin’s Function
The multimodal Rastrigin’s function is defined as  where and , with a global minimum of at . The and are chosen as 0.2domain and 0.8domain , respectively. Getting the global minimum of the Rastrigin’s function is a difficult process, as this test function has many local minima, which can be observed in its 3-dimensional surface plot in Figure 4. A more challenging 100-dimensional Rastrigin’s function is chosen in this case to corroborate the global searching ability of the proposed algorithm.
Figure 5 presents the optimization performances of the CSA and ACSA for Rastrigin’s function. It can be deduced from the figure that the proposed ACSA is superior to the standard CSA in terms of the convergence rate. It is evident that initially, the CSA appears to converge faster to the known global minimum than ASCA. However, as the evolution continues, the ACSA tends to get closer to the true solution than CSA. This is presumably due to the CSA is getting trapped in local solution, as there are many local minima present in this test function. Moreover, the optimization process of CSA may span a longer period of time than the proposed ACSA as it makes too small and cautious steps while exploring the search space, that is, the which controls the scale of the search patterns is specified as 1. On the contrary, the ACSA with adaptive step size control strategy performs more rigorous search through the solution space. The ACSA is able to find the global minimum in a mean of 20000 iterations. While on average, the CSA needs 2000 more iterations in order to reach the optimal solution.
4.5. Rosenbrock’s Function
Figure 6 illustrates the 3-dimensional surface plot for the Rosenbrock’s function, which is defined as : where and . This test function has a global minimum of at . The and are chosen as 0.02domain and 0.08domain , respectively. As shown in this figure, there is a long, narrow and parabolic shaped valley in the surface plot. This is the region where the global minimum is resided. Although finding this valley is not tedious, reaching the global optima is difficult .
The performances of the CSA and ACSA in terms of the convergence efficiency are shown in Figure 7. Comparing the best obtained results in among all the 30 trials, apparently that the proposed ACSA speeds up the computation more than one time faster, as compared to the standard CSA. Moreover, it can be noted that the average number of evaluations needed by the CSA is much inferior to the longest iteration achieved by the ACSA from among all the 30 independent runs.
4.6. Performance Comparison with Other Optimization Algorithms
In comparing the performances of different algorithms in optimizing the same benchmark functions, several ways with different stopping criteria can be considered, in which comparison of best fitness value for a fixed number of fitness function evaluations, a commonly used approach, has been adopted in this study. The other optimization algorithms considered are DE, evolutionary programming (EP), GA, PSO, simulated annealing (SA), and CSA. The simulations of all algorithms are performed for 30 independent runs with the number of fitness function evaluations is set to 2000. The best fitness value in each iteration is evaluated, and its average at each iteration from all the 30 trials is the measured. The obtained average best fitness values at fixed iteration number of 1, 500, 1000, 1500, and 2000 in optimizing the benchmark functions are summarized in Table 2.
As shown in this table, the proposed ACSA has higher precision than any of the other algorithms for all benchmarks considered, except for Griewank’s function. The DE produces solution that is closer to the global optima than the proposed ACSA in this case. However, it is pertinent to note that there is no or marginal improvement in DE after 500 function evaluations. The obtained best fitness value only decreases slightly from 0.35 to 0.33, which might indicates that the DE get trapped in local optima. In fact, as shown in Table 2, the DE, EP, and PSO algorithms usually converge faster initially, but they often get stuck in local optima easily, which is particularly obvious in the case of Ackley, de Jong, and Griewank’s functions. On the other hand, the proposed ACSA is getting closer to the global optima as the iteration increases gradually. The superiority of ACSA is more noticeable in the case of Ackley, de Jong, and Rastrigin’s function, where the ACSA reaches the near-optimal solution after 1500 function evaluations.
In this paper, by scrutinizing the advantages and the limitations of the standard CSA, a new modified CSA, specifically the ACSA, which adopts an adjustable step size in its computation, has been proposed. From all the considered benchmark optimization functions, the superiority of the ACSA over the standard CSA is validated, in terms of the convergence characteristics. The proposed algorithm preserves the uniqueness and the fascinating features of the original CSA, as both of the algorithms are able to find the global optimum when given enough computation time but the ACSA is able to converge faster in less iteration. This is probably due to its capability to concurrently refine the local search phase around the current good solutions while exploring more aggressively for the optimal solutions, attributed to the adopted adaptive step size. It is worth mentioning that through empirical simulations, increasing the maximum step size value will encourage a more thorough global exploration and eventually lead to faster convergence; however, the generated new solutions might fall outside the design domain for some cases. Thus, a moderate value is chosen in this study.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.
Financial support from Universiti Tun Hussein Onn Malaysia (UTHM) through Short Term Grant Vot 1287 is gratefully acknowledged.
J. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence, MIT Press, Cambridge, Mass, USA, 1992.
J. Kennedy and R. Eberhart, “Particle swarm optimization,” in Proceedings of the IEEE International Conference on Neural Networks Proceedings, vol. 1–6, pp. 1942–1948, 1995.View at: Google Scholar
R. Storn and K. Price, “Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces,” Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997.View at: Publisher Site | Google Scholar | MathSciNet
A. Colorni, M. Dorigo, and V. Maniezzo, “Distributed optimization by ant colonies,” in Proceedings of the 1st European Conference on Artificial Life, pp. 134–142, Paris, France, 1991.View at: Google Scholar
D. Karaboga, “An idea based on honey bee swarm for numerical optimization,” Tech. Rep. TR06, Erciyes University, Kayseri, Turkey, 2005.View at: Google Scholar
X. S. Yang, “Firefly algorithms for multimodal optimization,” in Stochastic Algorithms: Foundations and Applications, vol. 5792 of Lecture Notes in Computer Science, pp. 169–178, Springer, Berlin, Germany, 2009.View at: Publisher Site | Google Scholar | MathSciNet
X.-S. Yang and S. Deb, “Cuckoo search via Lévy flights,” in Proceedings of the World Congress on Nature and Biologically Inspired Computing (NABIC '09), pp. 210–214, December 2009.View at: Publisher Site | Google Scholar
X.-S. Yang and S. Deb, “Engineering optimisation by cuckoo search,” International Journal of Mathematical Modelling and Numerical Optimisation, vol. 1, no. 4, pp. 330–343, 2010.View at: Publisher Site | Google Scholar | Zentralblatt MATH
I. Durgun and A. R. Yildiz, “Structural design optimization of vehicle components using Cuckoo search algorithm,” Materials Testing, vol. 54, no. 3, pp. 185–188, 2012.View at: Publisher Site | Google Scholar
J. Piechocki, D. Ambroziak, A. Palkowski, and G. Redlarski, “Use of Modified Cuckoo Search algorithm in the design process of integrated power systems for modern and energy self-sufficient farms,” Applied Energy, vol. 114, pp. 901–908, 2013.View at: Publisher Site | Google Scholar
A. H. Gandomi, S. Talatahari, X.-S. Yang, and S. Deb, “Design optimization of truss structures using cuckoo search algorithm,” Structural Design of Tall and Special Buildings, vol. 22, pp. 1330–1349, 2013.View at: Publisher Site | Google Scholar
R. Sharma, K. P. S. Rana, and V. Kumar, “Performance analysis of fractional order fuzzy PID controllers applied to a robotic manipulator,” Expert Systems with Applications, vol. 41, no. 9, pp. 4274–4289, 2014.View at: Google Scholar
X.-T. Li and M.-H. Yin, “Parameter estimation for chaotic systems using the cuckoo search algorithm with an orthogonal learning method,” Chinese Physics B, vol. 21, no. 5, Article ID 050507, 2012.View at: Publisher Site | Google Scholar
M. Dhivya and M. Sundarambal, “Cuckoo Search for data gathering in wireless sensor networks,” International Journal of Mobile Communications, vol. 9, no. 6, pp. 642–656, 2011.View at: Publisher Site | Google Scholar
S. Goyal and M. S. Patterh, “Wireless sensor network localization based on cuckoo search algorithm,” Wireless Personal Communications, 2014.View at: Publisher Site | Google Scholar
A. Gandomi, X.-S. Yang, and A. H. Alavi, “Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems,” Engineering with Computers, vol. 29, no. 1, pp. 17–35, 2013.View at: Publisher Site | Google Scholar
A. H. Gandomi, X.-S. Yang, and A. H. Alavi, “Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems,” Engineering with Computers, vol. 29, no. 1, pp. 17–35, 2013.View at: Publisher Site | Google Scholar
A. K. Bhandari, V. K. Singh, A. Kumar, and G. K. Singh, “Cuckoo search algorithm and wind driven optimization based study of satellite image segmentation for multilevel thresholding using Kapur's entropy,” Expert Systems with Applications, vol. 41, pp. 3538–3560, 2014.View at: Publisher Site | Google Scholar
A. K. Bhandari, V. Soni, A. Kumar, and G. K. Singh, “Cuckoo search algorithm based satellite image contrast and brightness enhancement using DWT-SVD,” ISA Transactions, vol. 53, no. 4, pp. 1286–1296, 2014.View at: Publisher Site | Google Scholar
A. R. Yildiz, “Cuckoo search algorithm for the selection of optimal machining parameters in milling operations,” International Journal of Advanced Manufacturing Technology, vol. 64, no. 1–4, pp. 55–61, 2013.View at: Publisher Site | Google Scholar
K. Chandrasekaran and S. P. Simon, “Multi-objective scheduling problem: hybrid approach using fuzzy assisted cuckoo search algorithm,” Swarm and Evolutionary Computation, vol. 5, pp. 1–16, 2012.View at: Publisher Site | Google Scholar
S. Burnwal and S. Deb, “Scheduling optimization of flexible manufacturing system using cuckoo search-based approach,” The International Journal of Advanced Manufacturing Technology, vol. 64, no. 5-8, pp. 951–959, 2013.View at: Publisher Site | Google Scholar
M. K. Marichelvam, T. Prabaharan, and X. S. Yang, “Improved cuckoo search algorithm for hybrid flow shop scheduling problems to minimize makespan,” Applied Soft Computing Journal, vol. 19, pp. 93–101, 2014.View at: Google Scholar
S. Walton, O. Hassan, K. Morgan, and M. R. Brown, “Modified cuckoo search: a new gradient free optimisation algorithm,” Chaos, Solitons and Fractals, vol. 44, no. 9, pp. 710–718, 2011.View at: Publisher Site | Google Scholar
C. Li, J. Zhou, P. Kou, and J. Xiao, “A novel chaotic particle swarm optimization based fuzzy clustering algorithm,” Neurocomputing, vol. 83, pp. 98–109, 2012.View at: Publisher Site | Google Scholar
D. H. Ackley, “An empirical study of bit vector function optimization,” in Genetic Algorithms and Simulated Annealing, L. Davis, Ed., pp. 170–215, Morgan Kaufmann, Los Altos, Calif, USA, 1987.View at: Google Scholar
K. A. de Jong, An Analysis of the Behavior of a Class of Genetic Adaptive Systems, Department of Computer and Communication Sciences, University of Michigan, Ann Arbor, Mich, USA, 1975.
A. O. Griewank, “Generalized descent for global optimization,” Journal of Optimization Theory and Applications, vol. 34, no. 1, pp. 11–39, 1981.View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
L. A. Rastrigin, Extremal Control Systems, Nauka, Moscow, Russia, 1974.
H. H. Rosenbrock, “Automatic method for finding the greatest or least value of a function,” Engineering Technology & Applied Sciences, vol. 3, no. 3, pp. 175–184, 1960.View at: Publisher Site | Google Scholar