Journal of Applied Mathematics

Journal of Applied Mathematics / 2014 / Article

Research Article | Open Access

Volume 2014 |Article ID 402616 | 20 pages | https://doi.org/10.1155/2014/402616

Discrete and Continuous Optimization Based on Hierarchical Artificial Bee Colony Optimizer

Academic Editor: Roberto Natalini
Received11 Oct 2013
Revised23 Jan 2014
Accepted04 Feb 2014
Published17 Mar 2014

Abstract

This paper presents a novel optimization algorithm, namely, hierarchical artificial bee colony optimization (HABC), to tackle complex high-dimensional problems. In the proposed multilevel model, the higher-level species can be aggregated by the subpopulations from lower level. In the bottom level, each subpopulation employing the canonical ABC method searches the part-dimensional optimum in parallel, which can be constructed into a complete solution for the upper level. At the same time, the comprehensive learning method with crossover and mutation operator is applied to enhance the global search ability between species. Experiments are conducted on a set of 20 continuous and discrete benchmark problems. The experimental results demonstrate remarkable performance of the HABC algorithm when compared with other six evolutionary algorithms.

1. Introduction

Swarm intelligence (SI) is a new paradigm of artificial intelligence system whereby small entities are working together to produce an intelligent behaviour from simple rules [14]. Artificial bee colony algorithm (ABC) is one of the most popular members of the family of swarm intelligence, which simulates the social foraging behavior of a honeybee swarm [5, 6]. Due to its simple arithmetic and good robustness, the ABC algorithm has been widely used in solving many numerical optimizations [79] and engineering optimization problems [1014].

However, when solving complex multimodal problems, ABC algorithm suffers from the following drawbacks [8]. It is easily trapped into local minimums in the early generations, which leads to low population diversity in successive generations. With the dimension increasing, the information exchange of each individual is limited in a random dimension, resulting in a slow convergence rate. Due to the random selection of the neighbor bee and dimensions, food sources with higher fitness are not utilized, which influences the ability of global search.

Moreover, compared to the huge in-depth studies of other evolutionary and swarm intelligence algorithms, such as evolutionary algorithm (EA) [1518] and particle swarm optimization (PSO) [1923], how to improve the diversity of swarm or overcome the local convergence of ABC is still a challenging to the researchers in optimization domain.

In order to improve the performance of ABC on complex and high-dimensional problems, in this paper, a novel optimization algorithm called hierarchical ABC optimization (HABC) is proposed to extend the topology of original ABC algorithm from flat (one level) to hierarchical (multiple levels), which adopts multipattern cooperative evolutionary strategies.

It is noted that the hierarchical coevolutionary scheme has been incorporated in these intelligent algorithms. For instance, a hierarchical version of PSO was proposed in [24] on IEEE Trans on Evol Comp, in which a tree type hierarchy was incorporated in PSO. Chen et al. [25] proposed a hierarchical swarm model where each subswarm evolved based on original PSO. Peng and Lu [26] presented a similar PSO framework, in which several swarms evolve in parallel with mutation operator. This paper further extends the above studies based on hierarchical frameworks with more complex strategies, such as variables decomposing approach, random grouping of variables, and cross and mutation operation. In particular, instead of simply employing original SI algorithm in parallel like other hierarchical algorithms, the HABC employing the variables decomposing strategy with random grouping technique divides the complex vectors into smaller components assigned to different subswarms, which achieves a significant improvement of solving high-dimensional problem.

In terms of neighborhood concept, Kennedy and Mendes [27] analyzed effects of different topologies-based PSO algorithms. In 2004, a fully informed PSO using topologies and index-based neighborhood was proposed in [28]. Recently, Qu et al. [29] developed a neighborhood-based mutation strategy and integrated it with various niching DE algorithms, in which various topology types were considered. A distance-based locally informed particle swarm model was proposed in [30], which eliminates the need to specify any niching parameter and enhance the fine search ability of PSO. Obviously, many topology patterns can be used in different levels of our mode. In this work, we employ the most common topology, ring type, as the main structure of the proposed model.

The proposed HABC model is inherently different from others in the following aspects.(1)The cooperative coevolving approach based on divide-and-conquer strategy enhances the local search ability (exploitation); that is, by applying this method, the complex high-dimensional vectors can be decomposed into smaller components that are assigned to the lower hierarchy.(2)The traditional evolution operators, namely, the crossover and mutation, are applied to interaction of different species instead of the traditional individual exchange between populations. In this case, the neighbor bees with higher fitness can be chosen to crossover and mutation, which effectively enhances the global search ability and convergence speed to the global best solution (exploration).

Extensive studies based on a suit of 20 benchmark functions (including both continuous and discrete cases) are conducted to evaluate the performance of HABC. For comparison purposes, we also implemented the particle swarm optimization algorithm (PSO), cooperative particle swarm optimization algorithm (CPSO), artificial bee colony algorithm (ABC), cooperative artificial bee colony algorithm (CABC), covariance matrix adaptation evolution strategy (CMA-ES), and HABC variants on these functions. The experimental results are encouraging: the proposed HABC algorithm achieved remarkable search performance in terms of solution accuracy and convergence rate on almost all benchmark functions.

The rest of the paper is organized as follows. Section 2 describes the canonical ABC algorithm. In Section 3, the proposed hierarchical artificial bee colony (HABC) model is given. Section 4 tests the algorithm on the benchmarks and illustrates the results. Finally, Section 5 outlines the conclusions.

2. Canonical ABC Algorithm

The ABC algorithm is a relatively new SI algorithm by simulating the foraging behaviors of honey bee swarm, initially proposed by Karaboga and further developed by Basturk and Li et al. [8, 9, 31, 32]. In ABC, the colony of artificial bees are classified as three types: employed bees, onlookers, and scouts. Employed bees exploit the specific food sources; meanwhile, they pass the food information to onlooker bees. Onlooker bees choose good food sources based on the received information and then further exploit the food near their selected food source. The employed bee will become a scout when its food source has been abandoned. The fundamental mathematic representations are listed as follows.

In initialization phase, a group of food sources are generated randomly in the search space using the following equation: where and . SN is the number of food sources. is the number of variables, that is, problem dimension. and are the lower upper and upper bounds of the th variable, respectively.

In the employed bees’ phase, the neighbor food source (candidate solution) can be generated from the old food source of each employed bee in its memory using the following expression: where is a randomly selected food source and must be different from ; is a randomly chosen indexes; is a random number in range .

In the onlooker bees’ phase, a onlooker bee selects a food source depending on the probability value associated with that food source, and can be calculated as follows: where is the fitness value of th solution.

In scout bees’ phase, if a food source cannot be improved further through a predetermined cycle (called “limit” in ABC), the food source is supposed to be abandoned. The employed bee subsequently become a scout. A new food source will be produced randomly in the search space using (1).

The employed, onlooker, and scout bees’ phases will recycle until the termination condition is met.

3. Hierarchical Artificial Bee Colony Algorithm

The HABC integrates a two-level hierarchical coevolution scheme inspired by the concept and main ideas of multipopulation coevolution strategy and crossover and mutation operation. The flowchart of the HABC is shown in Figure 1. It includes four key strategy approaches: variables decomposing approach, random grouping of variables, background vector calculating approach, and crossover and mutation operation, which is presented as follows.

3.1. Hierarchical Multipopulation Optimization Model

As described in Section 2, we can see that the new food source is produced by a perturbation coming from a random single dimension in a randomly chosen bee. This causes an individual to may have discovered some good dimensions, while the other individuals that follow this bee are likely to choose worse vectors in -dimensions and abandon the good ones. On the other hand, when solving complex problems, single population-based artificial bee algorithms suffer from the following drawback of premature convergence at the early generations.

Hence, the HABC contains two levels, namely, the bottom level and top level, to balance exploring and exploiting ability. In Figure 1, in the bottom level, with the variables decomposing strategy, each subpopulation employs the canonical ABC method to search the part-dimensional optimum in parallel; that is, in each iteration, subpopulations in the bottom level generate best solutions, which are constructed into a complete solution species that are updated to the top level. In the top level, the multispecies community adopts the information exchange mechanism based on crossover operator, by which each species can learn from its neighborhoods in a specific topology. The vectors decomposing strategy and information exchange crossover operator can be described in detail as follows.

3.2. Variables Decomposing Approach

The purpose of this approach is to obtain finer local search in single dimensions inspired by the divide-and-conquer approach. Notice that two aspects must be analyzed: how to decompose the whole solution vector, and how to calculate the fitness of each individual of each subpopulation. The detailed procedure is presented as follows.

Step 1. The simplest grouping method is permitting a -dimensional vector to be split into subcomponents, each corresponding to a subpopulation of -dimensions, with individuals (where ). The th subpopulation is denoted as , .

Step 2. Construct complete evolving solution , which is the concatenation of the best subcomponents’ solutions by following: where represents the personal best solution of the th subpopulation.

Step 3. For each component , , do the following. (a)At employed bees’ phase, for each individual , , replace the th component of the by using the th component of individual . Caculate the new solution fitness: . If , then is replaced by .(b)Update positions by using (8).(c)At onlooker bees’ phase, repeat (a)-(b).

Step 4. Memorize the best solution achieved so far, and compare the best solution with and memorize the best one.

3.2.1. Random Grouping of Variables

To increase the probability of two interacting variables allocated to the same subcomponent, without assuming any prior knowledge of the problem, according to the random grouping of variables proposed by [21], we adopt the same random grouping scheme by dynamically changing group size. For example, for a problem of 100 dimensions, we can define that

Here, if we randomly decompose the -dimensional object vector into subcomponents at each iteration (i.e., we construct each of the subcomponents by randomly selecting -dimensions from the -dimensional object vector), the probability of placing two interacting variables into the same subcomponent becomes higher, over an increasing number of iterations [21].

3.3. The Information Exchange Mechanism Based on Crossover Operator between Multispecies

In the top level, we adopt crossover operator with a specific topology to enhance the information exchange between species, in which each species can learn from its symbiotic partner in the neighborhood. The key operations of this crossover procedure are described in Figure 2.

Step 5 (Select elites to the best-performing list (BPL)). First, a set of competent individuals from current species ’s neighborhood (i.e., ring topology) is selected to construct the best-performing list (BPL) with higher fitness that has larger probability to be selected. The size of BPL is equal with the number of current species . These individuals of BPL are regarded as elites. The selection operation tries to mimic the maturing phenomenon in nature, where the generated offspring will become more suitable to the environment by using these elites as parents.

Step 6 (Crossover and mutation between species). To produce well-performing individuals, parents are selected from the BPL’s elites only for the crossover operation. To select parents effectively, the tournament selection scheme is used. Firstly, two enhanced elites are selected randomly, and their fitness values are compared to select the elites. Then, the one with better fitness value is viewed as parent. Then, another parent is selected in the same way. Two offsprings are created by performing crossover on the selected parents. This paper adopts two representative crossover methods: single-point crossover and arithmetic crossover. For single-point crossover, the offspring is produced as the traditional GA crossover method [33]. For arithmetic crossover method, the offspring is produced by (6) as follows: where is newly produced offspring, and are randomly selected from BPL.

Step 7 (Update with greedy selection strategy). Not all current species are replaced by the elites from BPL; we set a selecting rate CR to determine the replaced individuals. Assuming that species size of is , then the replaced individuals number is . For the selected individual, , the newly produced offspring is then compared with , applying a greedy selection mechanism, in which the best one is remained. We can choose four selecting approaches: selecting the best individuals (i.e., individuals), a medium level of individuals, the worst individuals, and random individuals.

Hence, there are eight HABC variants according to different crossover and selection approaches, as listed in Table 1. In the next experiments, we will study the effect of different crossovers, selection modes, and CRs. With applying the effective social learning strategy, the information exchange between species is enhanced and the food sources with higher fitness are fully utilized.


AlgorithmsSelection methodsCrossover mode

HABC_SBSelect the best individualsSingle point
HABC_SWSelect the worst individualsSingle point
HABC_SMSelect medium individualsSingle point
HABC_SRSelect random individualsSingle point
HABC_ABSelect the best individualsArithmetic point
HABC_AWSelect the worst individualsArithmetic point
HABC_AMSelect medium individualsArithmetic point
HABC_ARSelect random individualsArithmetic point

In summary, in order to facilitate the below presentation and test formulation, we define a unified parameters for HABC model in Table 2. According to the process description as mentioned above, the flowchart of HABC algorithm is summarized in Figure 3, while the pseudocode for HABC algorithm is presented in Algorithm 1.



The number of species in top level
Subpopulation dividing D-dimensions into S-dimensions
Subpopulation size
Corresponding to a population of S-dimensions,
where
Dimensions of optimization problem
Top level population’s (species) ID counter from 1 to N
Button level population’s ID counter from 1 to K
The ith population (of the jth species)
CRSelection rate for replacing the offspring to the selected
individuals
The hierarchical interaction topology of HABC
The objective optimization goals

HABC algorithm
Set ;
INITIALIZE.
   Randomly divide the whole population into N species ( ) each possesses sub-populations ), each possesses bees;
Randomize ’s D-dimensions food source positions ; , , .
   Each sub-population with S dimensions (where S is randomly chosen from a set G, and ).
WHILE (the termination conditions are not met)
      for each species i,
      Initialize D-dimensions complete vector ( , ,
      which consists of the S-dimensions
      best solution .
      Randomly all D dimension indices;
      WHILE (the termination conditions are not met)
         for each sub-population do
            repeat
            Employed Bees’ Phase:
            For each employed bee
            Produce a new solution by using (2)
            Evaluate the new solution
            Apply Greedy selection choosing the better solution
                 end
                 Calculate the probability values Pi for the solution by using (2)
               Onlooker Bees’ Phase:
            for each employed bee
            Probabilistically choose a solution according to
            Produce a new solution by (2)
            Evaluate the new solution
            Apply Greedy selection choosing the better solution
                 end
            Re-initialize solutions not improved for Limit cycles
            Memorize the best solution
            for each individual of , do
                   Place best solution in the complete solution newGbest by:
                       , 
                   Update complete solution if it improves:
                   If (newGbest) <  f  (Gbest))
                      Then ·
                 end
         end
      end WHILE
      Select elites form neighborhood of
      BPL = the top best individuals of the ring topology
      Crossover and Mutation by (4)
      Update with applying Greedy selection mechanism from
      end
      find the global best solution gbest from the whole population P
      memorize the best solution of each
      Set ;
end WHILE

4. Experimental Study

In the experimental studies, to fully evaluate the performance of the HABC algorithm fairly, we employ a set of 20 benchmark functions as listed in the appendix, which can be classified as basic continuous benchmarks , CEC2005 benchmarks , and discrete benchmarks [34]. The number of function evaluations (FEs) is adopted as the time measure criterion substitute the number of iterations.

4.1. Experimental Settings

Eight variants of HABC based on different crossover methods and CR values were executed with six state-of-the-art EA and SI algorithms for comparisons: artificial bee colony algorithm (ABC) [4], cooperative artificial bee colony algorithm (CABC) [35], canonical PSO with constriction factor (PSO) [36], cooperative PSO (CPSO) [20], standard genetic algorithm (GA) [33], and covariance matrix adaptation evolution strategy (CMA-ES) [37].

CABC adopts the similar cooperative approach into original ABC algorithm. CPSO is a cooperative PSO model, cooperatively coevolving multiple PSO subpopulations. GA is the classical stochastic search technique mimicking the process of natural selection; the principle of CMA-ES is to apply the information of successful search steps to adjust the covariance matrix of the mutation distribution within an iterative procedure.

The population size and total generation number of all involved algorithms in this experiment are set as 50 and 100000. For the continuous testing functions used, the dimensions are all set as 100D. For the five discrete testing functions, the dimensions are set as Goldberg-30D, Bipolar-60D, Mulenbein-120D, Clerc’s problem 1-120D, and Clerc’s problem 2-120D.

According to the original literatures about the control parameters for the other algorithms involved, in continuous optimization experiment, the initialization conditions of CMA-ES are the same as in [37], and the number of offspring candidate solutions generated per time step is ; for ABC and CABC, the limit parameter is set to be , where is the objective function dimension and SN is the employed bees number. The split factor for CABC and CPSO is equal to the dimensions [20, 35]. For PSO and CPSO, the learning rates and were both set as 2.05 and the constriction factor .

In discrete optimization experiment, HABC is compared with the binary PSO version, ABC and GA. GA employs single point crossover operation (i.e., crossover rate is 0.8 and mutation rate is 0.01). For discrete HABC and discrete ABC, the update process used in the employ and onlooker stages is changed based on (1). The learning factor of producing new solution is calculated using (7). Then, the position update equation is defined by (8) for discrete problems. Discrete HABC variant adopts single-point crossover method. Consider

4.2. Parameter Sensitivity Analysis of HABC
4.2.1. Effects of Species Number

The species number of the top level in HABC needs to be tuned. Six continuous benchmark functions, Sphere 100D, Rosenbrock 100D, Rastrigin 100D, Schwefel 100D, Ackley 100D, and Griewank 100D and three discrete benchmark functions—Goldberg 30D, Bipolar 30D, and Clerc problem 1 120D—are employed to investigate the impact of this parameter. Set CR equal to 1 and all the functions run 30 sample times. As shown in Figure 4, it is visible that our proposed ABC got faster convergence rate and better optimal solutions on the involved test functions with increased . However, the performance improvement is not very remarkable using this parameter.

4.2.2. Choices of Crossover Mode and CR Value

The basic benchmark functions are adopted to evaluate the performance of HABC variants with different crossover modes andCRs. Four discrete benchmark functions are used to test the discrete versions of HABC_AB, HABC_AW, HABC_AM, and HABC_AR, which adopt single-point crossover mode.

Firstly, when CR is fixed at 0.2, all the functions are implemented for 30 times. From Table 3, we can observe that HABC_AW outperformed other variants on seven functions, except , while HABC_AM and HABC_AR get nearly the same results as HABC_AW on four functions. From the discrete functions results, HABC_SB markedly outperformed other variants, except . According to the rank listed in Table 3, the performances of the continuous algorithms involved are ordered as HABC_AW > HABC_AB > HABC_AR > HABC_AM > HABC_SR > HABC_SM > HABC_SB > HABC_SW and for the discrete versions are ordered as HABC_SB > HABC_SM > HABC_SW > HABC_SR.


FunctionHABC_SBHABC_SWHABC_SMHABC_SRHABC_ABHABC_AWHABC_AMHABC_AR

Mean
Std 000
Rank58674123
Mean 7.54 2.75
Std 8.69
Rank48762351
Mean
Std 000
Rank85764123
Mean 00000
Std 00000
Rank86711111
Mean 9.39 0000
Std 11.47 0000
Rank68571111
Mean
Std 0
Rank78546123
Mean
Std 0000
Rank68571111
Mean 00000
Std 00000
Rank86711111
Total Rank5257493920101514
Mean0.120.30.340.26
Std0.100.190.090.16
Rank1342
Mean0.070.250.210.26
Std0.080.180.730.13
Rank1324
Mean 0
Std 5.811.190
Rank4231
Mean1.943.612.723.86
Std0.590.950.901.10
Rank1324
Total Rank7111111

Then, the HABC_AW for continuous functions and HABC_SB for discrete functions are implemented to determine CR value. Form Tables 4 and 5, we can find that HABC variant with CR equal to 1 performs best on four functions among all five functions while CR equal to 0.05 gets best result on one function. According to the results with different CRs, we chose CR equal to 1 as an optimal value for the next experiments.


Function0.050.10.2.0.40.61

Mean 0 (64800)0 (56000)0  (36000)
Std 00000
Mean 5.50
Std 12.30
Mean 0 (69700)0 (49800)0  (38000)
Std 00000
Mean
Std
Mean0 (20000)0 (13500)0 (7900)0 (6860)0 (4400)0  (3100)
Std000000
Mean
Std
Mean (22000) (21500) (15400) (14220) (13800)   (12500)
Std000000
Mean0 (19000)0 (15800)0 (13000)0 (9800)0 (6100)0  (5400)
Std000000


Function0.050.10.20.40.61

Mean
Std
Mean 0
Std 0
Mean4.866.818.25
Std2.342.242.453.324.795.00
Mean6.701.691.753.615.288.73
Std 1.41
Mean 1.511.962.363.316.53
Std

4.2.3. Effects of Dynamically Changing Group Size

Obviously, the choice of value for split factor (i.e., subpopulation number) had a significant impact on the performance of the proposed algorithm. In order to vary during a run, we defined for 100D function optimization and set randomly to choose one element of . Then, the variant of HABC with dynamically changing is compared with that with fixed split number on four benchmark functions for 30 sample runs. From the results listed in Table 6 and Figure 5, we can observe that the performance is sensitive to the predefined value. HABC, with the dynamically changing , consistently obtained superior performance to other variants except . Moreover, it is not very easy to obtain the prior knowledge about the optimal value of the most real-world cases, so the random grouping scheme can be a suitable solution.


Function

Mean0  (91000)0  (45000)
Std00
Mean 1.471.56
Std 1.1 3
Mean00
Std00
Mean
Std
Mean
Std0