Journal of Applied Mathematics

Journal of Applied Mathematics / 2013 / Article
Special Issue

Harmony Search and Nature-Inspired Algorithms for Engineering Optimization

View this Special Issue

Research Article | Open Access

Volume 2013 |Article ID 139464 |

Mohammed Azmi Al-Betar, Ahamad Tajudin Khader, Mohammed A. Awadallah, Mahmmoud Hafsaldin Alawan, Belal Zaqaibeh, "Cellular Harmony Search for Optimization Problems", Journal of Applied Mathematics, vol. 2013, Article ID 139464, 20 pages, 2013.

Cellular Harmony Search for Optimization Problems

Academic Editor: Zong Woo Geem
Received08 Jul 2013
Revised15 Aug 2013
Accepted16 Aug 2013
Published28 Oct 2013


Structured population in evolutionary algorithms (EAs) is an important research track where an individual only interacts with its neighboring individuals in the breeding step. The main rationale behind this is to provide a high level of diversity to overcome the genetic drift. Cellular automata concepts have been embedded to the process of EA in order to provide a decentralized method in order to preserve the population structure. Harmony search (HS) is a recent EA that considers the whole individuals in the breeding step. In this paper, the cellular automata concepts are embedded into the HS algorithm to come up with a new version called cellular harmony search (cHS). In cHS, the population is arranged as a two-dimensional toroidal grid, where each individual in the grid is a cell and only interacts with its neighbors. The memory consideration and population update are modified according to cellular EA theory. The experimental results using benchmark functions show that embedding the cellular automata concepts with HS processes directly affects the performance. Finally, a parameter sensitivity analysis of the cHS variation is analyzed and a comparative evaluation shows the success of cHS.

1. Introduction

The optimization techniques have the utility of navigating the search space using effective operators driven by control parameters. The tricky point of the success of any optimization method is its ability to strike a suitable balance between exploration (diversification) and exploitation (intensification) of the problem search space [1]. Exploration is the optimization method capability of navigating a promising region of the search space, if necessary, while exploitation refers to the capability of fine-tuning the already-navigated regions to converge into the local optima [1].

Harmony search (HS) algorithm is a recent evolutionary algorithm (EA) proposed by Geem et al. [2] to imitate the musical improvisation process. Due to its advantages over other optimization methods, it stipulates fewer mathematical requirements in the initial search [3]. It has a novel stochastic derivative which reduces the number of iterations required to converge towards local minima [4], in addition to being simple, adaptable, general, and scalable [5]. Therefore, HS algorithm has been intensively tailored for several optimization problems such as timetabling [6, 7], nurse restoring [8], space allocation [9], and many others [1013]. However, due to the complex nature of some optimization problems and the avoidance of a premature convergence situation, HS theories are modified [14]. Furthermore, the control parameter adaptation for HS is also studied [5, 1517].

In a procedural context, HS, which is an iterative improvement algorithm, initiates with a population of random individuals stored in harmony memory (HM). At each iteration, a new individual is generated based on three operators: (i) memory consideration, which selects the variables of a new individual from whole HM individuals; (ii) pitch adjustment, which is responsible for local improvement, and (iii) random consideration, used to provide random elements for the new individual. The new individual is then evaluated and replaces the worst individual in HM, if it is better. This process is cycled until a desired condition is met (see Table 3).

As other EAs, HS algorithm interacts with whole individuals in the HM during each breeding step. The update process selects the worst individual from a single HM and replaces it with a new one, if better. The decentralized methods used in other structured EAs have shown that the performance of EA is improved. Examples include cellular genetic algorithm (cGA) [18], distributed EA (dEA) [19], cellular PSO [20], and others [21]. The main idea of these structured methods is to partition the population into several sets with common features [22].

Cellular genetic algorithm (cGA), in particular, is a decentralized method where the population is represented as a toroidal grid of two-dimensions, as shown in Figure 3 [23, 24]. The individuals are located in this toroidal in a predefined topology and solely interact with their nearest neighbors in the breeding step [22]. Note that all the neighborhoods have the same size and identical shape. This concept embedded in cGA provides useful advantages for the optimization domain [23] and parallel implementations [25, 26] because it assists in providing a high-level of diversity and yields a small diffusion of solutions through the search. The cGA provides a proper exploitation power inside each neighborhood of an individual by the operators of GA [22]. Theoretically, it has been shown that nonrandom mating keeps genetic diversity at higher level, thus preventing the algorithms from converging prematurely to the local optima [27].

The main objective of this paper is to embed the cellular automata concepts in the HS algorithm optimization framework, where the HM is arranged as a two-dimensional toroidal grid. The population diffusion will be expectably preserved, maintaining a high-level of diversity during the search, thus avoiding genetic drift. The improvisation process of HS algorithm is adjusted to interact with the neighborhoods of specific individuals. The updating process of HM is done within the neighborhoods of that individual. The results show that the new decentralized version of HS (i.e., cHS) algorithm improves the performance of HS using standard benchmark functions.

The rest of the paper is organized as follows. The basics of HS algorithm are described in Section 2. The proposed cellular harmony search (cHS) is discussed in Section 3. Results of the experiments are presented in Section 4. Finally, the conclusion and promising future research directions are provided in Section 5.

2. The Harmony Search Algorithm

The harmony search (HS) algorithm is a recent evolutionary approach proposed by Geem et al. [2]. It is initiated with a set of individuals stored in an augmented matrix called harmony memory (HM). It is a centralized algorithm where at each breeding step, it generates a new individual by interacting with the whole individuals in HM. HS follows three rules in the breeding step to generate a new individual: memory consideration, random consideration, and pitch adjustment. If the new individual is better than the worst individual in the whole HM, the replacement process is triggered. This process is repeated as many times as the HS is stagnated. The steps of HS algorithm are flowcharted in Figure 1, where each step is described below in more detail.

Step 1 (initialize parameters). The optimization problem is initially represented as , where is the objective function and is the set of decision variables. is the possible value range for each decision variable, where , and and are the lower and upper bounds for the decision variable , respectively, and is the number of decision variables. The parameters of the HS algorithm are also predefined in the following step.(a)The harmony memory consideration rate (HMCR), used in the breeding step to determine whether the value of a decision variable is to be selected from the individuals stored in the harmony memory (HM). (b)The harmony memory size (HMS) which determines the number of individuals in HM. (c)The pitch adjustment rate (PAR), which is used to decide the adjustments of some decision variables selected from memory. (d)The distance bandwidth (BW) which determines the distance of the adjustment that occurs to the individual in the pitch adjustment operator.(e)The number of improvisations (NI) which is similar to the number of generations.

Step 2 (initialize the harmony memory (HM)). The harmony memory (HM) is a matrix of size which includes sets of individuals determined by HMS (see (1)). In this step, these individuals are randomly generated as follows: , for all and for all , and generate a random number between 0 and 1. Consider

Step 3 (improvise a new individual). The HS algorithm generates a new individual, , using three operators: memory consideration, random consideration, and pitch adjustment.

Memory Consideration. In memory consideration, the value of the first decision variable in the new individual is randomly selected from the historical values, , stored in whole HM individuals. Values of the other decision variables, , are sequentially assigned in a similar way with a probability of HMCR, where . It is worth mentioning that this process interacts with the whole individuals in HM which might lead to a premature convergence situation due to the genetic drift.

Random Consideration. Decision variables that are not assigned with values according to memory consideration are randomly assigned according to their possible range by random consideration with a probability of (1-HMCR) as follows:

Pitch Adjustment. Each decision variable of a new individual, , that has been assigned a value by memory consideration is pitch adjusted with the probability of PAR, where as follows:

In pitch adjustment, if the decision for is Yes, the value of is modified to its neighboring value as follows: .

Step 4 (update HM). If the new individual, , has better objective function value than that of the worst individual stored in HM (i.e., in case HM is sorted), the worst individual will be replaced by the new one. Note that the worst individual is selected from the whole HM where the decentralized structure of the population is not observed.

Step 5 (check the termination rule). The HS algorithm will repeat Steps 3 and 4 of HS until a termination rule is met, which is normally decided by the value of NI parameter.

3. Cellular Harmony Search (cHS) Algorithm

There has been much interest from researchers and scientists from different fields exploiting the cellular automata (CA) in physics, biology, social science, computer science, and so on. The initial concepts of CA were developed by Neumann [28] and have been an effective research tool to be incorporated with a wide variety of disciplines.

The concepts of cellular automata (CA) are normally concerned with individual perspective. The main idea from CA is to provide a population of a particular structure formulated as a toroidal grid. The cell in the toroidal grid refers to an individual who communicates with his closest neighboring individuals so that all the individuals have exactly the same number of neighbors. This leads us to a kind of locality known as isolation by distance. Normally, the Manhattan distance is used to measure the distance between any individual and his neighbors. Note that the neighboring individuals have identical shapes and the same size [22].

There exist two-different kinds of cellular models based on how the breeding cycle is performed to the individuals. To put it differently, if the cycle is performed to the whole individuals at the same time, the cellular model is said to be synchronous, where the individuals of the next generation are simultaneously build. On the other hand, if the individuals of the population are sequentially updated with a particular order policy, an asynchronous cellular model is stated. For more discussion about the theory of cellular automata, relevant papers can be seen in [22, 29].

Cellular harmony search (cHS) algorithm can be considered as a new decentralized variation of HS which hinges on the structured HM. The individuals in the HM are arranged in the form of two-dimensional toroidal grid. This is meant to keep a high level of diversity during the breeding step and thus increases the chance to converge into global minima. This can be achieved by avoiding the genetic drift and providing a more suitable population diffusion during the search.

Some steps of cHS algorithm to the original version of HS algorithm presented in Section 2 have been adjusted. The adjustments are flowcharted in Figure 2. The breeding step of cHS solely interacts with the neighboring individuals of a randomly selected individual using “cellular memory consideration” operators. The update of HM step is adjusted to replace the worst individual amongst the neighboring individuals with a new individual, but not amongst the whole HM. Figure 3 shows the population on cellular structure (toroidal grid population), which inspires the concept of small neighborhood regions in the cellular automata. Particularly, the overlap of the neighborhood provides implicit technique migration into population. Therefore, the best solutions are diffused smoothly in the whole population, where the diversity of the cellular harmony search is preserved throughout the search. The cellular HS model includes several components.(1)Cell: the selected random individual in the population (the number of individual is HMS). (2)Cell space: the set of the whole individuals in HM. (3)Neighborhood: the set of potential mates of any individual.(4)Neighborhood shapes: the way of selecting the neighborhoods of the cell as seen in Figure 5.(5)Discrete time limit: the number of generations in HS algorithm which is normally determined by NI parameter.

The detailed steps of cHS are discussed in the steps below.

Step 1 (initialize cHS parameters and optimization problem). It is clear that the successful search of any metaheuristic method is based on skillful parameter setting. The parameters have different effects on optimization solutions. The parameters of cHS are harmony memory size (HMS), harmony memory consideration rate (HMCR), pitch adjusting rate (PAR), number of improvisation (NI), and the size of neighborhood (NH) determined by cellular structure (see Figure 5), where if , this means the neighbors structure is length with 8 neighbors.

Step 2 (initialize the harmony memory (HM)). This step is the same as Step 2 in the original version of HS. Note that the individuals of the HM are arranged as a two-dimensional toroidal grid as shown in Figure 4.

Step 3 (initialization of neighborhood matrix (NM)). NM is a binary matrix of size (see (4)). The binary values are assigned to each element based on NH parameter. This matrix is used during the breeding step to determine the neighboring individuals of any randomly selected individual. The NM matrix is filled by binary value as in (5). Consider

The is a set of all neighboring individuals of the individual arranged in two-dimensional toroidal mesh. This set is determined based on a neighborhood shape as seen in Figure 5.

Figure 4 shows how the HM individuals are mapped to toroidal grid. Note that the element reflects the index of the individual 1 in HM, while the reflects the individual index HMS in HM. The HMS value is the square value of K (i.e., ).

To map the element in toroidal grid Y and the individual index in HM, the following will be used:

To map the index of the individual in HM to the element () in the matrix NM, the following will be used:

This mapping mechanism between the individual index of HM and the elements in Y is very useful to determine the neighboring of any individual in HM. As shown in Figure 5, there are several neighborhood shapes to determine the neighbors of any individual. For example, the L5 neighborhood shape takes the nearest neighbors of a given cell axial direction. Therefore, to determine the individual indexes in HM that belong to the neighbors of the individual index using L5, the and should be calculated using (7) and (8). And the set of neighbors of individual called , then (8) is used to map the elements in Y to the corresponding individual indexes in HM. The same step is used if different neighborhood shapes (i.e., L9, C9,…) are used to determine the neighboring individuals of .

As can be noted, the individuals in the set will be assigned by 1 in NM for the individual while others will be assigned by 0.

Step 4 (select random individual from HM). The selection of a random individual , , is done to determine the neighboring individuals taken from the matrix NM  (i.e., ), where is the number of the neighboring individuals; that is, . The neighboring individuals interact together in order to generate the new individual in the next step.

Step 5 (generate a new individual). In this step, a new individual, , is generated based on the three operators: cellular memory consideration, pitch adjustment, and random consideration. The whole process is drawn in Figure 6.

The pitch adjustment and random consideration operators in cHS algorithm are the same as those in the original version of HS algorithm. However, the memory consideration is modified to be inline with the concepts of cellular automata as follows.

Cellular Memory Consideration. As in cellular GA, the cellular memory consideration solely interacts with the close individuals of taken from the set of . The other individuals are not used in the breeding step. The value of the first decision variable is randomly selected from the historical values, , stored individuals in the set . The other decision variables, , are sequentially assigned in a similar way with a probability of HMCR, where .

It is worth mentioning that cellular memory consideration is able to control the diffusion between the individuals in HM, and thus, it is able to preserve the cHS diversity as long as the search process is iterated. By this strategy, the population is structured and it is possible to improve the numerical behavior of the cHS algorithm.

Step 6 (update the harmony memory). This step is modified in cHS algorithm. The worst individual from the set of neighbors , that is, , is replaced by the new individual , if better. Note that the replacement process is done taking into account the neighboring individuals in the set only.

Step 7 (check stopping criterion). In this step, the cHS will stop if the maximum number of the iteration (i.e., NI) is reached; otherwise the algorithm repeats Steps 4 to 6. The pseudocode of cHS is demonstrated in Algorithm 1.

Set HMCR, PAR, NI, HMS, bw.
, , , ;
Calculate , ;
Generate(NM) {generate the neighborhood matrix}
While     do
 select random individual
for     do
  if     then
    ; {Memory Consideration}
   if     then
     ;  {Pitch Adjustment}
   end if
    ;  {Random Consideration}
  end if
end for
 find( ) where
if     then
  include to HM;
  exclude from HM;
end if
end while

4. Experimental and Comparative Evaluation

In this section, cHS algorithm is evaluated using benchmark functions circulated in the literature used to evaluate different variations of HS algorithm. The comparative evaluation is demonstrated while the sensitivity analysis of the control parameters of the proposed method is carried out.

4.1. Comparative Evaluation

In this section, a set of test functions designed for the special session on real-parameter optimization organized in the 2005 IEEE Congress on Evolutionary Computation (CEC 2005) [30], is used. The CEC 2005 comprises 25 test functions including 5 unimodals and 20 multimodals functions, as shown in Table 1. Note that a full discussion about these functions can be taken from Sunganthan et al. [30]. The CEC 2005 provides a suitable number of comparative methods in which the proposed method can be evaluated, as abbreviated in Table 2.

Abb. Function name Search range

Shifted sphere function −450
Shifted Schwefel's problem 1.2 −450
Shifted rotated high conditioned elliptic function −450
Shifted Schwefel's problem 1.2 with noise in fitness −450
Schwefel's problem 2.6 with global optimum on bounds −310
Shifted Rosenbrock's function 390
Shifted rotated Griewank's function without bounds −180
Shifted rotated Ackley's function with global optimum on bounds −140
Shifted rastrigin's function −330
Shifted rotated Rastrigin's function −330
Shifted rotated Weierstrass function 90
Schwefel's problem 2.13 −460
Expanded extended Griewank's plus Rosenbrock's function (F8F2) −130
Shifted rotated expanded Scaffer's F6 −300
Hybrid composition function 120
Rotated hybrid composition function 120
Rotated hybrid composition function with noise in fitness 120
Rotated hybrid composition function 10
Rotated hybrid composition function with a narrow basin for the global optimum 10
Rotated hybrid composition function with the global optimum on the bounds 10
Rotated hybrid composition function 360
Rotated hybrid composition function with high condition number matrix 360
Noncontinuous rotated hybrid composition function 360
Rotated hybrid composition function 260
Rotated hybrid composition function without bounds 260

Key Method name Reference

BLXGL50 Hybrid real-coded genetic algorithms with female and male differentiation [31]
BLX-MA Adaptive local search parameters for real-coded memetic algorithms [32]
CoEVO Real-parameter optimization using the mutation step coevolution [33]
DE Real-parameter optimization with differential evolution [34]
DMS-L-PSO Dynamic multiswarm particle swarm optimizer with local search [35]
EDA Experimental results for the special session on real-parameter optimization at CEC’2005: a simple, continuous EDA [36]
K-PCX A population-based, steady-state procedure for real-parameter optimization [37]
G-CMA-ES A restart CMA evolution strategy with increasing population size [38]
L-CMA-ES Performance evaluation of an advanced local search evolutionary algorithm [39]
L-SaDE Self-adaptive differential evolution algorithm [40]
SPC-PNX Real-parameter optimization performance study on the CEC-2005 benchmark with SPC-PNX [41]