Research Article | Open Access
Discrete and Continuous Optimization Based on Hierarchical Artificial Bee Colony Optimizer
This paper presents a novel optimization algorithm, namely, hierarchical artificial bee colony optimization (HABC), to tackle complex high-dimensional problems. In the proposed multilevel model, the higher-level species can be aggregated by the subpopulations from lower level. In the bottom level, each subpopulation employing the canonical ABC method searches the part-dimensional optimum in parallel, which can be constructed into a complete solution for the upper level. At the same time, the comprehensive learning method with crossover and mutation operator is applied to enhance the global search ability between species. Experiments are conducted on a set of 20 continuous and discrete benchmark problems. The experimental results demonstrate remarkable performance of the HABC algorithm when compared with other six evolutionary algorithms.
Swarm intelligence (SI) is a new paradigm of artificial intelligence system whereby small entities are working together to produce an intelligent behaviour from simple rules [1–4]. Artificial bee colony algorithm (ABC) is one of the most popular members of the family of swarm intelligence, which simulates the social foraging behavior of a honeybee swarm [5, 6]. Due to its simple arithmetic and good robustness, the ABC algorithm has been widely used in solving many numerical optimizations [7–9] and engineering optimization problems [10–14].
However, when solving complex multimodal problems, ABC algorithm suffers from the following drawbacks . It is easily trapped into local minimums in the early generations, which leads to low population diversity in successive generations. With the dimension increasing, the information exchange of each individual is limited in a random dimension, resulting in a slow convergence rate. Due to the random selection of the neighbor bee and dimensions, food sources with higher fitness are not utilized, which influences the ability of global search.
Moreover, compared to the huge in-depth studies of other evolutionary and swarm intelligence algorithms, such as evolutionary algorithm (EA) [15–18] and particle swarm optimization (PSO) [19–23], how to improve the diversity of swarm or overcome the local convergence of ABC is still a challenging to the researchers in optimization domain.
In order to improve the performance of ABC on complex and high-dimensional problems, in this paper, a novel optimization algorithm called hierarchical ABC optimization (HABC) is proposed to extend the topology of original ABC algorithm from flat (one level) to hierarchical (multiple levels), which adopts multipattern cooperative evolutionary strategies.
It is noted that the hierarchical coevolutionary scheme has been incorporated in these intelligent algorithms. For instance, a hierarchical version of PSO was proposed in  on IEEE Trans on Evol Comp, in which a tree type hierarchy was incorporated in PSO. Chen et al.  proposed a hierarchical swarm model where each subswarm evolved based on original PSO. Peng and Lu  presented a similar PSO framework, in which several swarms evolve in parallel with mutation operator. This paper further extends the above studies based on hierarchical frameworks with more complex strategies, such as variables decomposing approach, random grouping of variables, and cross and mutation operation. In particular, instead of simply employing original SI algorithm in parallel like other hierarchical algorithms, the HABC employing the variables decomposing strategy with random grouping technique divides the complex vectors into smaller components assigned to different subswarms, which achieves a significant improvement of solving high-dimensional problem.
In terms of neighborhood concept, Kennedy and Mendes  analyzed effects of different topologies-based PSO algorithms. In 2004, a fully informed PSO using topologies and index-based neighborhood was proposed in . Recently, Qu et al.  developed a neighborhood-based mutation strategy and integrated it with various niching DE algorithms, in which various topology types were considered. A distance-based locally informed particle swarm model was proposed in , which eliminates the need to specify any niching parameter and enhance the fine search ability of PSO. Obviously, many topology patterns can be used in different levels of our mode. In this work, we employ the most common topology, ring type, as the main structure of the proposed model.
The proposed HABC model is inherently different from others in the following aspects.(1)The cooperative coevolving approach based on divide-and-conquer strategy enhances the local search ability (exploitation); that is, by applying this method, the complex high-dimensional vectors can be decomposed into smaller components that are assigned to the lower hierarchy.(2)The traditional evolution operators, namely, the crossover and mutation, are applied to interaction of different species instead of the traditional individual exchange between populations. In this case, the neighbor bees with higher fitness can be chosen to crossover and mutation, which effectively enhances the global search ability and convergence speed to the global best solution (exploration).
Extensive studies based on a suit of 20 benchmark functions (including both continuous and discrete cases) are conducted to evaluate the performance of HABC. For comparison purposes, we also implemented the particle swarm optimization algorithm (PSO), cooperative particle swarm optimization algorithm (CPSO), artificial bee colony algorithm (ABC), cooperative artificial bee colony algorithm (CABC), covariance matrix adaptation evolution strategy (CMA-ES), and HABC variants on these functions. The experimental results are encouraging: the proposed HABC algorithm achieved remarkable search performance in terms of solution accuracy and convergence rate on almost all benchmark functions.
The rest of the paper is organized as follows. Section 2 describes the canonical ABC algorithm. In Section 3, the proposed hierarchical artificial bee colony (HABC) model is given. Section 4 tests the algorithm on the benchmarks and illustrates the results. Finally, Section 5 outlines the conclusions.
2. Canonical ABC Algorithm
The ABC algorithm is a relatively new SI algorithm by simulating the foraging behaviors of honey bee swarm, initially proposed by Karaboga and further developed by Basturk and Li et al. [8, 9, 31, 32]. In ABC, the colony of artificial bees are classified as three types: employed bees, onlookers, and scouts. Employed bees exploit the specific food sources; meanwhile, they pass the food information to onlooker bees. Onlooker bees choose good food sources based on the received information and then further exploit the food near their selected food source. The employed bee will become a scout when its food source has been abandoned. The fundamental mathematic representations are listed as follows.
In initialization phase, a group of food sources are generated randomly in the search space using the following equation: where and . SN is the number of food sources. is the number of variables, that is, problem dimension. and are the lower upper and upper bounds of the th variable, respectively.
In the employed bees’ phase, the neighbor food source (candidate solution) can be generated from the old food source of each employed bee in its memory using the following expression: where is a randomly selected food source and must be different from ; is a randomly chosen indexes; is a random number in range .
In the onlooker bees’ phase, a onlooker bee selects a food source depending on the probability value associated with that food source, and can be calculated as follows: where is the fitness value of th solution.
In scout bees’ phase, if a food source cannot be improved further through a predetermined cycle (called “limit” in ABC), the food source is supposed to be abandoned. The employed bee subsequently become a scout. A new food source will be produced randomly in the search space using (1).
The employed, onlooker, and scout bees’ phases will recycle until the termination condition is met.
3. Hierarchical Artificial Bee Colony Algorithm
The HABC integrates a two-level hierarchical coevolution scheme inspired by the concept and main ideas of multipopulation coevolution strategy and crossover and mutation operation. The flowchart of the HABC is shown in Figure 1. It includes four key strategy approaches: variables decomposing approach, random grouping of variables, background vector calculating approach, and crossover and mutation operation, which is presented as follows.
3.1. Hierarchical Multipopulation Optimization Model
As described in Section 2, we can see that the new food source is produced by a perturbation coming from a random single dimension in a randomly chosen bee. This causes an individual to may have discovered some good dimensions, while the other individuals that follow this bee are likely to choose worse vectors in -dimensions and abandon the good ones. On the other hand, when solving complex problems, single population-based artificial bee algorithms suffer from the following drawback of premature convergence at the early generations.
Hence, the HABC contains two levels, namely, the bottom level and top level, to balance exploring and exploiting ability. In Figure 1, in the bottom level, with the variables decomposing strategy, each subpopulation employs the canonical ABC method to search the part-dimensional optimum in parallel; that is, in each iteration, subpopulations in the bottom level generate best solutions, which are constructed into a complete solution species that are updated to the top level. In the top level, the multispecies community adopts the information exchange mechanism based on crossover operator, by which each species can learn from its neighborhoods in a specific topology. The vectors decomposing strategy and information exchange crossover operator can be described in detail as follows.
3.2. Variables Decomposing Approach
The purpose of this approach is to obtain finer local search in single dimensions inspired by the divide-and-conquer approach. Notice that two aspects must be analyzed: how to decompose the whole solution vector, and how to calculate the fitness of each individual of each subpopulation. The detailed procedure is presented as follows.
Step 1. The simplest grouping method is permitting a -dimensional vector to be split into subcomponents, each corresponding to a subpopulation of -dimensions, with individuals (where ). The th subpopulation is denoted as , .
Step 2. Construct complete evolving solution , which is the concatenation of the best subcomponents’ solutions by following: where represents the personal best solution of the th subpopulation.
Step 3. For each component , , do the following. (a)At employed bees’ phase, for each individual , , replace the th component of the by using the th component of individual . Caculate the new solution fitness: . If , then is replaced by .(b)Update positions by using (8).(c)At onlooker bees’ phase, repeat (a)-(b).
Step 4. Memorize the best solution achieved so far, and compare the best solution with and memorize the best one.
3.2.1. Random Grouping of Variables
To increase the probability of two interacting variables allocated to the same subcomponent, without assuming any prior knowledge of the problem, according to the random grouping of variables proposed by , we adopt the same random grouping scheme by dynamically changing group size. For example, for a problem of 100 dimensions, we can define that
Here, if we randomly decompose the -dimensional object vector into subcomponents at each iteration (i.e., we construct each of the subcomponents by randomly selecting -dimensions from the -dimensional object vector), the probability of placing two interacting variables into the same subcomponent becomes higher, over an increasing number of iterations .
3.3. The Information Exchange Mechanism Based on Crossover Operator between Multispecies
In the top level, we adopt crossover operator with a specific topology to enhance the information exchange between species, in which each species can learn from its symbiotic partner in the neighborhood. The key operations of this crossover procedure are described in Figure 2.
Step 5 (Select elites to the best-performing list (BPL)). First, a set of competent individuals from current species ’s neighborhood (i.e., ring topology) is selected to construct the best-performing list (BPL) with higher fitness that has larger probability to be selected. The size of BPL is equal with the number of current species . These individuals of BPL are regarded as elites. The selection operation tries to mimic the maturing phenomenon in nature, where the generated offspring will become more suitable to the environment by using these elites as parents.
Step 6 (Crossover and mutation between species). To produce well-performing individuals, parents are selected from the BPL’s elites only for the crossover operation. To select parents effectively, the tournament selection scheme is used. Firstly, two enhanced elites are selected randomly, and their fitness values are compared to select the elites. Then, the one with better fitness value is viewed as parent. Then, another parent is selected in the same way. Two offsprings are created by performing crossover on the selected parents. This paper adopts two representative crossover methods: single-point crossover and arithmetic crossover. For single-point crossover, the offspring is produced as the traditional GA crossover method . For arithmetic crossover method, the offspring is produced by (6) as follows: where is newly produced offspring, and are randomly selected from BPL.
Step 7 (Update with greedy selection strategy). Not all current species are replaced by the elites from BPL; we set a selecting rate CR to determine the replaced individuals. Assuming that species size of is , then the replaced individuals number is . For the selected individual, , the newly produced offspring is then compared with , applying a greedy selection mechanism, in which the best one is remained. We can choose four selecting approaches: selecting the best individuals (i.e., individuals), a medium level of individuals, the worst individuals, and random individuals.
Hence, there are eight HABC variants according to different crossover and selection approaches, as listed in Table 1. In the next experiments, we will study the effect of different crossovers, selection modes, and CRs. With applying the effective social learning strategy, the information exchange between species is enhanced and the food sources with higher fitness are fully utilized.
In summary, in order to facilitate the below presentation and test formulation, we define a unified parameters for HABC model in Table 2. According to the process description as mentioned above, the flowchart of HABC algorithm is summarized in Figure 3, while the pseudocode for HABC algorithm is presented in Algorithm 1.
4. Experimental Study
In the experimental studies, to fully evaluate the performance of the HABC algorithm fairly, we employ a set of 20 benchmark functions as listed in the appendix, which can be classified as basic continuous benchmarks , CEC2005 benchmarks , and discrete benchmarks . The number of function evaluations (FEs) is adopted as the time measure criterion substitute the number of iterations.
4.1. Experimental Settings
Eight variants of HABC based on different crossover methods and CR values were executed with six state-of-the-art EA and SI algorithms for comparisons: artificial bee colony algorithm (ABC) , cooperative artificial bee colony algorithm (CABC) , canonical PSO with constriction factor (PSO) , cooperative PSO (CPSO) , standard genetic algorithm (GA) , and covariance matrix adaptation evolution strategy (CMA-ES) .
CABC adopts the similar cooperative approach into original ABC algorithm. CPSO is a cooperative PSO model, cooperatively coevolving multiple PSO subpopulations. GA is the classical stochastic search technique mimicking the process of natural selection; the principle of CMA-ES is to apply the information of successful search steps to adjust the covariance matrix of the mutation distribution within an iterative procedure.
The population size and total generation number of all involved algorithms in this experiment are set as 50 and 100000. For the continuous testing functions used, the dimensions are all set as 100D. For the five discrete testing functions, the dimensions are set as Goldberg-30D, Bipolar-60D, Mulenbein-120D, Clerc’s problem 1-120D, and Clerc’s problem 2-120D.
According to the original literatures about the control parameters for the other algorithms involved, in continuous optimization experiment, the initialization conditions of CMA-ES are the same as in , and the number of offspring candidate solutions generated per time step is ; for ABC and CABC, the limit parameter is set to be , where is the objective function dimension and SN is the employed bees number. The split factor for CABC and CPSO is equal to the dimensions [20, 35]. For PSO and CPSO, the learning rates and were both set as 2.05 and the constriction factor .
In discrete optimization experiment, HABC is compared with the binary PSO version, ABC and GA. GA employs single point crossover operation (i.e., crossover rate is 0.8 and mutation rate is 0.01). For discrete HABC and discrete ABC, the update process used in the employ and onlooker stages is changed based on (1). The learning factor of producing new solution is calculated using (7). Then, the position update equation is defined by (8) for discrete problems. Discrete HABC variant adopts single-point crossover method. Consider
4.2. Parameter Sensitivity Analysis of HABC
4.2.1. Effects of Species Number
The species number of the top level in HABC needs to be tuned. Six continuous benchmark functions, Sphere 100D, Rosenbrock 100D, Rastrigin 100D, Schwefel 100D, Ackley 100D, and Griewank 100D and three discrete benchmark functions—Goldberg 30D, Bipolar 30D, and Clerc problem 1 120D—are employed to investigate the impact of this parameter. Set CR equal to 1 and all the functions run 30 sample times. As shown in Figure 4, it is visible that our proposed ABC got faster convergence rate and better optimal solutions on the involved test functions with increased . However, the performance improvement is not very remarkable using this parameter.
4.2.2. Choices of Crossover Mode and CR Value
The basic benchmark functions are adopted to evaluate the performance of HABC variants with different crossover modes andCRs. Four discrete benchmark functions are used to test the discrete versions of HABC_AB, HABC_AW, HABC_AM, and HABC_AR, which adopt single-point crossover mode.
Firstly, when CR is fixed at 0.2, all the functions are implemented for 30 times. From Table 3, we can observe that HABC_AW outperformed other variants on seven functions, except , while HABC_AM and HABC_AR get nearly the same results as HABC_AW on four functions. From the discrete functions results, HABC_SB markedly outperformed other variants, except . According to the rank listed in Table 3, the performances of the continuous algorithms involved are ordered as HABC_AW > HABC_AB > HABC_AR > HABC_AM > HABC_SR > HABC_SM > HABC_SB > HABC_SW and for the discrete versions are ordered as HABC_SB > HABC_SM > HABC_SW > HABC_SR.
Then, the HABC_AW for continuous functions and HABC_SB for discrete functions are implemented to determine CR value. Form Tables 4 and 5, we can find that HABC variant with CR equal to 1 performs best on four functions among all five functions while CR equal to 0.05 gets best result on one function. According to the results with different CRs, we chose CR equal to 1 as an optimal value for the next experiments.
4.2.3. Effects of Dynamically Changing Group Size
Obviously, the choice of value for split factor (i.e., subpopulation number) had a significant impact on the performance of the proposed algorithm. In order to vary during a run, we defined for 100D function optimization and set randomly to choose one element of . Then, the variant of HABC with dynamically changing is compared with that with fixed split number on four benchmark functions for 30 sample runs. From the results listed in Table 6 and Figure 5, we can observe that the performance is sensitive to the predefined value. HABC, with the dynamically changing , consistently obtained superior performance to other variants except . Moreover, it is not very easy to obtain the prior knowledge about the optimal value of the most real-world cases, so the random grouping scheme can be a suitable solution.