A Novel Self-Adaptive Harmony Search Algorithm

Abstract

The harmony search algorithm is a music-inspired optimization technology and has been successfully applied to diverse scientific and engineering problems. However, like other metaheuristic algorithms, it still faces two difficulties: parameter setting and finding the optimal balance between diversity and intensity in searching. This paper proposes a novel, self-adaptive search mechanism for optimization problems with continuous variables. This new variant can automatically configure the evolutionary parameters in accordance with problem characteristics, such as the scale and the boundaries, and dynamically select evolutionary strategies in accordance with its search performance. The new variant simplifies the parameter setting and efficiently solves all types of optimization problems with continuous variables. Statistical test results show that this variant is considerably robust and outperforms the original harmony search (HS), improved harmony search (IHS), and other self-adaptive variants for large-scale optimization problems and constrained problems.

1. Introduction

Harmony search (HS) is a relatively new metaheuristic optimization algorithm [1] that imitates the improvisation process of Jazz musicians as shown in Figure 1. Each musician is analogous to each decision variable; the audience’s aesthetics are analogous to the objective function; the pitch range of a musical instrument corresponds to the value range of the decision variables; the musicians’ improvisations correspond to local and global search schemes; musical harmony at certain time corresponds to a solution vector at certain iteration. In general, a musician’s improvisation follows three rules [2]: (1) playing any one pitch from his/her memory, (2) playing an adjacent pitch to one pitch from his/her memory, and (3) playing a random pitch from the possible sound range. The HS algorithm mimics these rules and converts them into corresponding evolutionary strategies: (1) choosing any one value from the HS memory (defined as memory considerations), (2) choosing a value adjacent to one from the HS memory (defined as pitch adjustments), and (3) choosing a random value from the possible value range (defined as randomisation). The evolutionary rules in the HS algorithm are effectively directed using the following parameters: (1) harmony memory size (HMS, i.e., the number of solutions in harmony memory), (2) harmony memory considering rate (HMCR, distinctly HMCR∈[0,1]), (3) pitch adjusting rate (PAR, distinctly PAR∈[0,1]), (4) fine-tuning bandwidth (Bw, i.e., the adjustment range of a variable), and (5) the number of improvisations (NI, i.e., the maximum number of iterations).

Figure 1 

The Analogy between Improvisation and Optimization (from Geem [3]).

The values of these parameters, particularly three key parameters, HMCR, PAR, and Bw, affect the performances of HS. However, parameter setting is problem dependent and difficult, like other evolutionary computational methods, such as genetic algorithms (GA), simulated annealing (SA), particle swarm optimization (PSO), and ant colony optimization (ACO). To alleviate the burden of manually finding the best parameter values, this work proposes a natural and novel approach to parameter setting based on the improvisation practice of musicians.

For any algorithm that involves the operation of searching randomly, the most difficult is to find an appropriate balance between diversity and intensity in the process of searching. when emphasising diversity, it is easy to conduct a slower convergence and even divergence; when emphasising intensity, it is easy to conduct premature convergence and become trapped in a local optimal solution. The HS algorithm is no exception. Thus, this paper also proposes a novel self-adaptive search mechanism.

The rest of this paper is organized as follows. Section 2 presents a concise review of related previous work. Section 3 describes the novel self-adaptive harmony search (NSHS) algorithm. Section 4 introduces a test suite composed of twelve benchmark functions. Section 5 analyzes the robustness of the proposed variant. Section 6 confirms the superiority of the novel variant by the comparison test. The last section makes a conclusion.

2. Review of Related Previous Work

2.1. Original Harmony Search Algorithm

The original HS algorithm developed by [1] is carried out as follows.

Step 1 (initialise the problem and algorithm parameters). First, the optimization problem is specified as follows: min_𝐱𝑓(𝐱)s.t.𝑥_𝑖∈𝑙𝑏_𝑖,𝑢𝑏_𝑖(𝑖=1,2,…,𝑛), (1)
where 𝑓(𝐱) is the objective function, 𝐱 is a vector consisting of 𝑛 decision variables, and 𝑙𝑏_𝑖 and 𝑢𝑏_𝑖 are the lower and upper bounds of variable 𝑥_𝑖, respectively. The original HS algorithm basically considers the objective function only. If the problem has equality and/or inequality constraints and a solution vector violates any of them, either the resulting infeasible solution will be abandoned or its objective function value will have a certain penalty score added.
Second, the parameters HMS, HMCR, PAR, Bw, and NI must be valued, and the initial harmony memory (HM) must be generated. Consider 𝑥HM=¹¹𝑥¹₂⋯𝑥¹_𝑛𝑓𝑥¹𝑥²₁𝑥²₂⋯𝑥²_𝑛𝑓𝑥²𝑥⋮⋮⋮⋮⋮_HMS1𝑥^HMS₂⋯𝑥^HMS_𝑛𝑓𝑥^HMS. (2)

Step 2 (improve a new harmony vector by improvisation). A new solution 𝑥^new=(𝑥^new₁,𝑥^new₂,…,𝑥^new_𝑛), termed the harmony vector, is generated based on memory considerations, pitch adjustments, and randomisation. Consider 𝑥^new_𝑖⟵𝑥_𝑖∈𝑥¹_𝑖,𝑥²_𝑖,…,𝑥^HMS_𝑖()𝑥;withprobabilityHMCR∗1−PAR_𝑖⋅()[]𝑥1+𝑟∗Bw;withprobability𝑟∼U−1,1HMCR∗PAR_𝑖∈𝑙𝑏_𝑖,𝑢𝑏_𝑖;withprobability1−HMCR. (3)

Step 3 (assess the new harmony vector and update the harmony memory). In this step, the merit of the new harmony vector is assessed based on its objective function value. If the new vector is better than the worst harmony vector in the current HM, the new vector is included in the HM and the existing worst vector is excluded from the HM.

Step 4 (check the stopping criterion). Repeat Steps 2~3 until the termination criterion is satisfied; that is, the number of iterations reaches NI.
The structure of the HS algorithm is much easier. Its core evolutionary process includes only three operations: choosing, adjusting, and randomising. However, the meta-heuristic algorithm handles intensification and diversification. Its intensification and diversification are represented by HMCR and PAR, respectively, and its superiority is confirmed by the existence of a large number of successful applications to diverse scientific and engineering problems, including timetabling [4], the 0-1 knapsack problem [5], power system design [6], transmission network planning [7], flow-shop scheduling [8], structural design [9], water network design [10], soil stability analysis [11], ecological conservation [12], and vehicle routing [13].
However, the major disadvantage of the HS algorithm is that its performance depends on the parameter setting and static parameter values are not conducive to balancing intensification and diversification.

2.2. Variants Based on Parameter Setting

To improve the performance of the original HS algorithm or alleviate the burden of manually finding the best parameter values, some variants based on parameter setting have been proposed.

2.2.1. Improved Harmony Search (IHS) Proposed by Mahdavi et al. [14]

In this variant, PAR and Bw are dynamically adjusted by the following formulae: PAR(𝑘)=PAR_min+PAR_max−PAR_min∗𝑘,(𝑘)NIBw=Bw_max𝑘∗expNI∗lnBw_minBw_max, (4)(5)

where PAR_min and PAR_max indicate the minimum and maximum pitch adjusting rates, respectively. 𝑘 and NI denote the 𝑘th-iteration and the maximum number of iterations, respectively. Bw_min, and Bw_max indicate the minimum and maximum bandwidths.

In fact, this variant does not alleviate the difficulty of parameter setting, because users have to provide suitable values of PAR_min, PAR_max, Bw_min and Bw_max. Furthermore, using (4), the pitch adjusting rate gradually increases from PAR_min to PAR_max through an iterative process, which is rather inconsistent with the observation that a larger iteration number usually generates a more perfect harmony and requires a smaller tuning probability.

Later, Omran and Mahdavi [15] abandoned the Bw parameter by replacing direct adjustment with the use of a random variable for the best harmony vector as shown in (6), calling this new variant the global-best harmony search (GHS). Consider 𝑥^new_𝑖⟵𝑥^best_𝑘(𝑖=1,2,…,𝑛;𝑘∈{1,2,…,𝑛})withprobabilityHMCR⋅PAR. (6)

As Wang and Huang [16] noted, the name of this variant is misused and the method can easily cause premature convergence. In fact, the most serious defect in the GHS variant is that if the domain of each variable is different, that is, [𝑙𝑏_𝑖,𝑢𝑏_𝑖]≠[𝑙𝑏_𝑘,𝑢𝑏_𝑘],(𝑖,𝑘=1,2,…,𝑛;𝑖≠𝑘), then the infeasible solution will frequently be generated. Thus, Pan et al. [17] corrected this serious drawback using the following equation: 𝑥^new_𝑖⟵𝑥^best_𝑖(𝑖=1,2,…,𝑛)withprobabilityHMCR⋅PAR. (7)

2.2.2. The Self-Adaptive Harmony Search Proposed by Wang and Huang [16]

Wang and Huang recognised the shortcomings of IHS and GHS. They used the same modification to dynamically adapt PAR values during the search process, but the value of PAR is decreased linearly with time. Thus, their variant needs are still the values of PAR_min and PAR_max. The innovation of the variant is to adjust the value of each variable according to the maximal and minimal values in the harmony memory (HM) and without the Bw parameter again. Consider 𝑥^new_𝑖⟵𝑥_𝑖+𝑟⋅max_{[ℎ∈1,HMS]}HM(ℎ,𝑖)−𝑥_𝑖𝑥;if𝑟≥0.5_𝑖−𝑟⋅𝑥_𝑖−min_{[ℎ∈1,HMS]}()[].HMℎ,𝑖;if𝑟<0.5𝑟∼U0,1 (8)

Although this adjustment enables each variable to avoid violating its boundary constraint, the bandwidth is larger; therefore, the convergence is very slow and can even stop.

2.2.3. The Self-Adaptive Harmony Search Proposed by Pan et al

Pan et al. [18] proposed a local-best harmony search algorithm with dynamic subpopulations (DLHS). In the variant, the whole harmony memory is equally divided into m subgroups. Moreover, to keep population diversity, the small-sized sub-HMs are regrouped once the period 𝑅 is reached. In the variant, HMCR and PAR are from a parameter set list (PSL) with length Len, which is randomly generated by the initial value with a uniform distribution in the range PAR∈[0,1] and HMCR∈[0.9,1] and then updated once the list is empty. In the variant, the value of the parameter Bw is also dynamically changed: Bw(𝑘)=Bw_max−Bw_max−Bw_minNI∗2𝑘;if2𝑘<NI,Bw_min;otherwise. (9)

Although the variant takes the advantage of each local optimum, it consumes too much time as a cost. Thus, Pan et al. [17] proposed a self-adaptive global best harmony search algorithm (SGHS) based on the GHS algorithm. In the new variant, the value of HMCR (PAR) is normally distributed in the range [0.9,1]([0,1]) with mean HMCRm (PARm) and standard deviation 0.01 (0.05). The initial values of HMCRm and PARm are given by the user. In a fixed learning period (LP), the values of HMCRm and PARm do not change; after LP, the values are recalculated by averaging all HMCR and PAR values recorded during the previous LP. In the variant, Bw is still determined by (9).

Obviously, the two variants require extra jobs to set the values of more parameters than the basic HS algorithm and other variants. Moreover, it is irrational that the value of PAR still varies in the interval [0,1] regardless of the number of iterations because a successful search should gradually converge and the value of PAR should be decreased with time to prevent oscillation.

These defects inspired me to investigate how the key parameters of the HS algorithm do not depend on the problem’s feature again.

2.3. Variants Based on Hybridisation with Other Metaheuristics

To improve the performance of the original HS, many researchers have hybridised the algorithm with other meta-heuristics, such as GA, PSO, and ACO [19]. This hybridisation can be categorised into two approaches [20]: (1) integrating some components of other meta-heuristic algorithms into the HS structure, and conversely, (2) integrating some HS components into other meta-heuristic algorithm structures. Perhaps hybrid methods are better than single meta-heuristics.

However, in my opinion, hybridisation will depart from the essential nature of the original heuristic and the resulting mimicry will also be nondescript. Thus, the correct way to investigate this should be by refining the essence of the imitated objects. This idea inspired me to improve the search mechanism of the original HS algorithm by capturing the deep essence of musical improvisation.

3. A Novel Self-Adaptive Harmony Search Algorithm

3.1. Design of the Control Parameters

The HS algorithm includes five parameters: harmony memory size (HMS), harmony memory considering rate (HMCR), pitch adjusting rate (PAR), fine-tuning bandwidth (Bw), and the number of improvisations (NI). Among these parameters, HMCR, PAR, and Bw are crucial for the convergence of HS.

It should be noted that each improvisation of a musician is always accompanied by fine-tuning until the improvisation is finished. That is, it is necessary to adjust each variable value before the HS algorithm is ended. Therefore, it is reasonable that the value of PAR ought to equal one and senseless to consider this parameter again. For this reason, the proposed self-adaptive variant does not use the PAR parameter.

The value of HMCR in the basic HS algorithm is fixed at its initialisation and does not change until the search ends. As the value of HMCR decreases, harmony memory is used less efficiently, the algorithm converges much more slowly, and more time is consumed. As the value of HMCR increases, harmony memory is used more efficiently, the method converges more rapidly, less time is consumed, and the HS is easily trapped in a local optimum. To enhance the power of the HS algorithm, HMCR is usually valued in the interval [0.9,0.99]. However, the reason for this is not revealed [21]. In fact, musicians depend on their memory more for intricate than simple music. This fact inspired me to construct the HMCR parameter based on the dimension of the problem to be solved. That is, the dimension of the problem is analogous to the complexity of the music, 1HMCR=1−,𝑛+1 (10)

where 𝑛 denotes the dimension of the problem to be solved. Clearly, 0.5 < HMCR < 1; as the number of dimensions of the problem increases, the value of HMCR increases and harmony memory is used more frequently, and vice versa.

It should be noted that musicians fine-tune their playing throughout their improvisation, and the tuning range is wider at the beginning and narrower as the music is completed. This phenomenon inspired me to construct a dynamic fine-tuning bandwidth 𝑏𝑤_𝑖 for each variable 𝑥_𝑖, with bounds 𝑙𝑏_𝑖 and 𝑢𝑏_𝑖 and iteration number 𝑘: 𝑏𝑤_𝑖(𝑘)=𝑢𝑏_𝑖−𝑙𝑏_𝑖∗𝑘1001−()NI𝑘=1,2,…,NI;if𝑓std>0.00010.0001;otherwise, (11)

where 𝑓std=std(𝑓(𝐱¹),𝑓(𝐱²),…,𝑓(𝐱^HMS)) denotes the standard deviation for the objective function values in the current HM. The value of 𝑏𝑤_𝑖 decreases gradually as the iteration number increases and increases as the range of each decision variable increases. The domain of each decision variable corresponds to the pitch range of each musical instrument; the gradual decrease in the adjustment range as the iteration number increases corresponds to the gradual decrease of fine-tuning bandwidth with increasing improvisation number. (𝑢𝑏−𝑙𝑏)/100 is analogous to the unit of fine-tuning bandwidth, and 0.0001 represents the minimal adjustment. 𝑓std≤0.0001 represents the slight difference between the merit of each harmony vector in HM. In fact, at the late stages of musical improvisation, the music is gradually mellowed and the fine-tuning range used is very narrow.

3.2. Design of the Self-Adaptive Mechanism

For every meta-heuristic algorithm, accelerating the convergence and preventing the premature convergence appear contradictory. To resolve this contradiction, the evolutionary mechanism of the HS algorithm has to be properly revised. It should be again noted that fine-tuning occurs throughout the whole process of musician improvisation and the fine-tuning range is wider at the early stage and narrower at later stages. Therefore, this phenomenon can be mimicked by the following evolutionary operations: 𝑥^′_𝑖⟵𝑥^𝑗_𝑖+𝑟2∗𝑏𝑤_𝑖;𝑗∈{1,2,…,HMS}if𝑟<HMCR𝑙𝑏_𝑖+𝑟1∗𝑢𝑏_𝑖−𝑙𝑏_𝑖if𝑟≥HMCR&+𝑟2∗𝑏𝑤_𝑖;𝑓std>0.0001min_𝑗𝑥^𝑗_𝑖+𝑟1∗max_𝑗𝑥^𝑗_𝑖−min_𝑗𝑥^𝑗_𝑖if𝑟≥HMCR&+𝑟2∗𝑏𝑤_𝑖;𝑓std≤0.0001, (12)

where random numbers 𝑟,𝑟1∼U[0,1];𝑟2∼U[−1,1]. The standard deviation for the objective function values fstd represents the whole merit of the new harmony vectors in the current HM. If the standard deviation is less than a given threshold 0.0001, that is, if 𝑓std≤0.0001, then the difference between the function values is very small. If the iteration number is sufficiently large, then the distribution of the harmony vectors is centralised; thus, a new random value should be generated within the narrow range [min_𝑗𝑥^𝑗_𝑖,max_𝑗𝑥^𝑗_𝑖] for rapid convergence.

3.3. Evolutionary Process of the NSHS Algorithm

On the basis of the novel parameter-setting method and self-adaptive evolutionary mechanism, the computational procedure of the proposed variant can be given, as shown in Algorithm 1.

NSHS Algorithm Initialise the problem and algorithm parameters:    Define a standard fitness function 𝑓(𝑥),𝑥𝑖∈[𝑙𝑏𝑖,𝑢𝑏𝑖](𝑖=1,2,…,𝑛) by (1)    Define the parameters HMCR by (10), Bw by (11), HMS and NI    Generate randomly an initial HM and calculate fstd Search the best solution: While (𝑘≤NI) do   Generate a new harmony vector by the following loop   While (𝑖≤𝑛) do     First, generate a new value for every variable by the following rules:     If (𝑟≤HMCR,𝑟∼𝑈[0,1]), then choose uniformly a value from the current memory for variable      𝑥′𝑖=𝑥𝑗𝑖;𝑗∈{1,2,…,HMS}     Else, generate randomly         If 𝑓std>0.0001, then generate randomly in the range of [𝑙𝑏𝑖,𝑢𝑏𝑖]        𝑥′𝑖=𝑙𝑏𝑖+𝑟1∗𝑢𝑏𝑖−𝑙𝑏𝑖;𝑟1∼𝑈[0,1]         Else, generate randomly in the range of [min𝑗𝑥𝑗𝑖,max𝑗𝑥𝑗𝑖]        𝑥′𝑖=min𝑗𝑥𝑗𝑖+𝑟1∗(max𝑗𝑥𝑗𝑖−min𝑗𝑥𝑗𝑖);𝑟1∼𝑈[0,1]         End if     End if     Second, adjust the new value of every variable by the following rules:     If 𝑓std>0.0001, then        𝑥𝑖′′=𝑥′+𝑢𝑏𝑖−𝑙𝑏𝑖∗𝑘1001−NI∗𝑟2;𝑟2∼𝑈[−1,1]     Else, generate randomly in the range of [min𝑗𝑥𝑗𝑖,max𝑗𝑥𝑗𝑖]        𝑥𝑖′′=𝑥′𝑖+0.0001∗𝑟2;𝑟2∼𝑈[−1,1]     End if   End while   Assess the merit of the new harmony vector and update the current harmony memory   If 𝑓(𝑥new)<𝑓(𝑥worst)=max𝑗𝑓(𝑥𝑗), then     Accept the new harmony vector and replace the worst vector in the HM with it.     Recalculate the standard deviation of the fitness function values fstd.   End if End while Determine the optimal solution in the HM by the minimum fitness.

Algorithm 1 

Pseudo code of the NSHS algorithm.

4. Benchmark Functions

This section lists the global minimization benchmark functions used to evaluate the performances of the proposed algorithm. The test suite comprises six well-known unimodal benchmark functions, six well-known multimodal benchmark functions, and six constrained optimization functions. To evaluate the robustness of a stochastic search algorithm, the twelve unconstrained benchmark functions are randomly shifted and orthogonally rotated, and four benchmark functions are with normal noise in fitness.

4.1. Unimodal Benchmark Functions

Table 1 summarizes the shifted rotated unimodal benchmark functions. Figure 2 depicts their 3D shapes if the dimension is equal to two.

Table 1 

Six shifted rotated unimodal benchmark functions.

(a) 2D Cigar-Tablet's function

(b) 2D Rosenbrock's function

(c) 2D Schwefel's problem 1.2

(d) 2D Schwefel’s problem 2.21

(e) 2D Schwefel’s problem 2.22

(f) 2D Sphere function

Figure 2 

Six well-known unimodal benchmark function landscapes.

4.2. Multimodal Benchmark Functions

Table 2 summarizes the shifted rotated multimodal benchmark functions. Figure 3 depicts their 3D shapes if the dimension is equal to two.

Table 2 

Six well-known multimodal benchmark functions.

(a) 2D Ackley's function

(b) 2D Bohachevsky's function

(c) 2D Griewank's function

(d) 2D Rastrigin's function

(e) 2D Schaffer's function

(f) 2D Schwefel’s problem 2.13

Figure 3 

Six well-known multimodal benchmark function landscapes.

4.3. Constrained Benchmark Functions

The six constrained optimization functions have also been described in the literature [2]. Thus, we only give their corresponding penalty functions defined in this study. It should be noted that these penalty functions are composed of many penalty terms in order to precisely control every violation: 𝑓₁₃=min𝑥₁−2²+𝑥₂−1²,𝑥₁,𝑥₂∈[−10,10]. (13)

If max(abs(𝑥)−10)>0, then 𝑓₁₃+max(abs(𝑥)−10)↦𝑓₁₃.

If 𝑥₁−2𝑥₂+1=𝑔₁(𝑥)≠0, then 𝑓₁₃+10abs(𝑔₁)↦𝑓₁₃.

If −𝑥²₁/4−𝑥²₂+1=𝑔₂(𝑥)<0, then 𝑓₁₃−10𝑔₂↦𝑓₁₃.

For 𝑓₁₃, the known best solution 𝑥^∗=(0.8229,0.9114) and 𝑓^∗=1.3935. Consider 𝑓₁₄=min𝑥²₁+𝑥₂−11²+𝑥₁+𝑥²₂−7²,𝑥₁,𝑥₂∈[0,6]. (14)

If max(𝑥−6)>0, then 𝑓₁₄+max(𝑥−6)↦𝑓₁₄.

If max(−𝑥)>0, then 𝑓₁₄+max(−𝑥)↦𝑓₁₄.

If 4.84−(𝑥₁−0.05)²−(𝑥₂−2.5)²=𝑔₁(𝑥)<0, then 𝑓₁₄−100𝑔₁↦𝑓₁₄.

If 𝑥²₁+(𝑥₂−2.5)²−4.84=𝑔₂(𝑥)<0, then 𝑓₁₄−𝑔₂↦𝑓₁₄.

For 𝑓₁₄, the known best solution 𝑥^∗=(2.2468,2.3819) and 𝑓^∗=13.5909. Consider 𝑓₁₅=min5.357847𝑥²₃+0.8356891𝑥₁𝑥₂+37.293239𝑥₁,𝑥−40792.141₁∈[78,102],𝑥₂∈[33,45],𝑥₃,𝑥₄,𝑥₅∈[27,45]. (15)

If 𝑥₁<78, then abs(𝑓₁₅)−𝑥₁+78↦𝑓₁₅.

If 𝑥₁>102, then abs(𝑓₁₅)+𝑥₁−102↦𝑓₁₅.

If 𝑥₂<33, then abs(𝑓₁₅)−𝑥₂+33↦𝑓₁₅.

If 𝑥₂>45, then abs(𝑓₁₅)+𝑥₂−45↦𝑓₁₅.

If 27−max(𝑥₃,𝑥₄,𝑥₅)>0, then abs(𝑓₁₅)−max(𝑥₃,𝑥₄,𝑥₅)+27↦𝑓₁₅.

If max(𝑥₃,𝑥₄,𝑥₅)−45>0, then abs(𝑓₁₅)+max(𝑥₃,𝑥₄,𝑥₅)−45↦𝑓₁₅.

If 85.334407+0.0056858𝑥₂𝑥₅+0.0006262𝑥₁𝑥₄−0.002205𝑥₃𝑥₅=𝑔₁(𝑥)<0, then abs(𝑓₁₅)−𝑔₁↦𝑓₁₅.

If 6.66593−0.0056858𝑥₂𝑥₅−0.0006262𝑥₁𝑥₄+0.002205𝑥₃𝑥₅=𝑔₂(𝑥)<0, then abs(𝑓₁₅)−10𝑔₂↦𝑓₁₅.

If −9.48751+0.0071317𝑥₂𝑥₅+0.0029955𝑥₁𝑥₂+0.0021813𝑥²₃=𝑔₃(𝑥)<0, then abs(𝑓₁₅)−𝑔₃↦𝑓₁₅.

If 29.48751−0.0071317𝑥₂𝑥₅−0.0029955𝑥₁𝑥₂−0.0021813𝑥²₃=𝑔₄(𝑥)<0, then abs(𝑓₁₅)−𝑔₄↦𝑓₁₅.

If −10.699039+0.0047026𝑥₃𝑥₅+0.0012547𝑥₁𝑥₃+0.0019085𝑥₃𝑥₄=𝑔₅(𝑥)<0 then abs(𝑓₁₅)−10000𝑔₅↦𝑓₁₅.

If 15.699039−0.0047026𝑥₃𝑥₅−0.0012547𝑥₁𝑥₃−0.0019085𝑥₃𝑥₄=𝑔₆(𝑥)<0, then abs(𝑓₁₅)−𝑔₆↦𝑓₁₅.

For 𝑓₁₅, the known best solution 𝑥^∗=(78,33,29.995,45,36.7764) and 𝑓^∗=−30665.5. Consider 𝑓₁₅=min𝑥₁−10²+5𝑥₂−12²+𝑥⁴₃+3𝑥₄−11²+10𝑥⁶₅+7𝑥²₆+𝑥⁴₇−4𝑥₆𝑥₇−10𝑥₆−8𝑥₇,𝑥_𝑖∈[−10,10]. (16)

If max(abs(𝑥)−10)>0, then 𝑓₁₆+max(abs(𝑥)−10)↦𝑓₁₆.

If 127−2𝑥²₁−3𝑥⁴₂−𝑥₃−4𝑥²₄−5𝑥₅=𝑔₁(𝑥)<0, then 𝑓₁₆−100𝑔₁↦𝑓₁₆.

If 282−7𝑥₁−3𝑥₂−10𝑥²₃−𝑥₄+𝑥₅=𝑔₂(𝑥)<0, then 𝑓₁₆−𝑔₂↦𝑓₁₆.

If 196−23𝑥₁−𝑥²₂−6𝑥²₆+8𝑥₇=𝑔₃(𝑥)<0, then 𝑓₁₆−𝑔₃↦𝑓₁₆.

If −4𝑥²₁−𝑥²₂+3𝑥₁𝑥₂−2𝑥²₃−5𝑥₆+11𝑥₇=𝑔₄(𝑥)<0, then 𝑓₁₆−100𝑔₄↦𝑓₁₆.

For 𝑓₁₆, the known best solution 𝑥^∗=(2.3305,1.9513,−0.4775,4.3657, −0.6245,1.0381,1.5942) and 𝑓^∗=680.6301. Consider 𝑓₁₇=min𝑥₁+𝑥₂+𝑥₃,𝑥₁∈[],𝑥100,10000₂,𝑥₃∈[1000,10000],𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑥₈∈[10,1000]. (17)

If 𝑥₁<100, then 𝑓₁₇+100(100−𝑥₁)↦𝑓₁₇.

If 𝑥₁>10000, then 𝑓₁₇+𝑥₁−10000↦𝑓₁₇.

If 1000−max(𝑥₂,𝑥₃)>0, then 𝑓₁₇+100(1000−max(𝑥₂,𝑥₃))↦𝑓₁₇.

If max(𝑥₂,𝑥₃)−10000>0, then 𝑓₁₇+max(𝑥₂,𝑥₃)−10000↦𝑓₁₇.

If 10−max(𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑥₈)>0, then 𝑓₁₇+10−max(𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑥₈)↦𝑓₁₇.

If max(𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑥₈)−1000>0, then 𝑓₁₇+max(𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑥₈)−1000↦𝑓₁₇.

If 1−0.0025(𝑥₄+𝑥₆)=𝑔₁(𝑥)<0, then 𝑓₁₇−3500𝑔₁↦𝑓₁₇.

If 1−0.0025(𝑥₅+𝑥₇−𝑥₄)=𝑔₂(𝑥)<0, then 𝑓₁₇−6000𝑔₂↦𝑓₁₇.

If 1−0.01(𝑥₈−𝑥₅)=𝑔₃(𝑥)<0, then 𝑓₁₇−4000𝑔₃↦𝑓₁₇.

If 𝑥₁𝑥₆−833.33252𝑥₄−100𝑥₁+83333.333=𝑔₄(𝑥)<0, then 𝑓₁₇−𝑔₄↦𝑓₁₇.

If 𝑥₂𝑥₇−1250𝑥₅−𝑥₂𝑥₄+1250𝑥₄=𝑔₅(𝑥)<0, then 𝑓₁₇−0.1𝑔₅↦𝑓₁₇.

If 𝑥₃𝑥₈−𝑥₃𝑥₅+2500𝑥₅−1250000=𝑔₆(𝑥)<0, then 𝑓₁₇−𝑔₆↦𝑓₁₇.

For 𝑓₁₇, the known best solution 𝑥^∗=(579.3167,1359.943,5110.071,182.0174,295.5985, 217.9799,286.4162,395.5979) and 𝑓^∗=7049.3309. Consider 𝑓₁₈=min𝑥²₁+𝑥²₂+𝑥₁𝑥₂−14𝑥₁−16𝑥₂+𝑥₃−10²+4𝑥₄−5²+𝑥₅−3²+2𝑥₆−1²+5𝑥²₇+7𝑥₈−11²+2𝑥₉−10²+𝑥₁₀−7²,𝑥+45_𝑖∈[−10,10](𝑖=1,2,…,10). (18)

If max(abs(𝑥)−10)>0, then 𝑓₁₈+max(abs(𝑥)−10)↦𝑓₁₈.

If 105−4𝑥₁−5𝑥₂+3𝑥₇−9𝑥₈=𝑔₁(𝑥)<0, then 𝑓₁₈−10𝑔₁↦𝑓₁₈.

If −10𝑥₁+8𝑥₂+17𝑥₇−2𝑥₈=𝑔₂(𝑥)<0, then 𝑓₁₈−10𝑔₂↦𝑓₁₈.

If 8𝑥₁−2𝑥₂−5𝑥₉+2𝑥₁₀+12=𝑔₃(𝑥)<0, then 𝑓₁₈−10𝑔₃↦𝑓₁₈.

If −3(𝑥₁−2)²−4(𝑥₂−3)²−2𝑥²₃+7𝑥₄+120=𝑔₄(𝑥)<0, then 𝑓₁₈−10𝑔₄↦𝑓₁₈.

If −5𝑥²₁−8𝑥₂−(𝑥₃−6)²+2𝑥₄+40=𝑔₅(𝑥)<0, then 𝑓₁₈−10𝑔₅↦𝑓₁₈.

If −𝑥²₁−2(𝑥₂−2)²+2𝑥₁𝑥₂−14𝑥₅+6𝑥₆=𝑔₆(𝑥)<0, then 𝑓₁₈−10𝑔₆↦𝑓₁₈.

If −0.5(𝑥₁−8)²+2(𝑥₂−4)²−3𝑥²₅+𝑥₆+30=𝑔₇(𝑥)<0, then 𝑓₁₈−10𝑔₇↦𝑓₁₈.

If 3𝑥₁−6𝑥₂−12(𝑥₉−8)²+7𝑥₁₀=𝑔₈(𝑥)<0, then 𝑓₁₈−𝑔₈↦𝑓₁₈.

For 𝑓₁₈, the known best solution 𝑥^∗=(2.1720,2.3637,8.7739,5.0960,0.9907,1.4306, 1.3216, 9.8287,8.2801,8.3760)and 𝑓^∗=24.3062.

5. Robustness of the Proposed NSHS Algorithm

The performance of a stochastic search algorithm often is affected by the initial populations. For the HS algorithm, the initial population distribution is determined by the value of HMS and the initial search ranges. Thus, this section investigates the robustness of the proposed variant from the two aspects.

5.1. Effect of HMS

Yang [21] declared that the value of HMS is insensitive to the performance of the original HS algorithm. To reveal the fact that the value of HMS does not also have a significant impact on the performance of the NSHS, a full-factorial experiment was conducted. In the experiment, the HMS is valued at four levels (5, 10, 20, and 50) as a fixed factor; above twelve unconstrained benchmark functions with 10-dimension and six constrained functions were employed; the simulation was repeated 30 times, and each simulation underwent 50,000 evaluations for the objective function. The relative error between the running result and the global or known optimum is tested as a dependent variable. The descriptive statistical result of these errors is shown in Table 3.

Table 3 

The descriptive statistical result of the relative errors of the NSHS algorithm in the case of HMS = 5, 10, and 50, respectively.

Clearly, there exists the tiny difference. However, we can claim that the difference is due to the function tested rather than the HMS by the test result of univariate analysis of variance, seeing the report of SPSS shown in Table 4. Therefore, the performance of the proposed NSHS algorithm is not affected by the value of HMS. To save storage space, it is encouraged to use a small HMS, as Pan et al. [17] explained that “since HM is emulating musician’s short-term memory and the short-term memory of the human is known to be small, it is logical.”

Table 4 

The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and HMS).

5.2. Effect of Initial Search Ranges

To identify whether the performance of the proposed variant depends on the initial HM, a full-factorial experiment was conducted using the parameters HMS = 10 and NI = 50,000. In the experiment, the initial HM was randomly generated at three types of the initial search ranges, that is, the whole domain, the left part, and the right part. The experiment was repeated 30 times and 1620 observations were collected. The descriptive statistical result of the relative errors of the proposed variant is shown in Table 5.

Table 5 

The descriptive statistical result of the relative errors of the NSHS algorithm using three types of the initial search ranges.

Obviously, there still exists the small difference. However, the test report of SPSS shown in Table 6 still indicates that the difference is due to the function tested rather than the initial search ranges.

Table 6 

The report of SPSS on the univariate (relative error) analysis of variance with two fixed factors (problem/function and initial range).

By the results of analysis of variance as shown in Tables 4 and 6, the performance of the proposed NSHS algorithm does not depend on the initial HM. By the descriptive statistical results as shown in Tables 3 and 5, moreover, no matter how the initial HM is, the NSHS algorithm can stability search for the global optima of these benchmark functions with a tiny error, except for function f₂. Therefore, the novel variant is considerably robust.

6. Superiority of the Proposed NSHS Algorithm

To confirm the significant superiority of the proposed variant, a comparison test was conducted. The test compared the performance of the NSHS with the original HS algorithm [1] and the other variants mentioned in Section 2, including Mahdavi’s IHS [14], Wang’s self-adaptive variant represented by HSw for convenience [16], Pan’s DLHS [18], and SGHS [17]. For fair comparison, the parameter settings are those proposed by their developers, as shown in Table 7.

Table 7 

Algorithm parameters.

For sufficiently persuasive comparative results, every algorithm was run independently 30 times to solve the aforementioned 18 benchmark functions on the same computer (CPU: AMD 4 × 3.5 Ghz; RAM: 4 GB), and, at the beginning of solving a problem, each algorithm used the same initial HM. The conclusion is made based on an analysis of variance rather than a pairwise 𝑡-test because Garc’s study yet indicated that the latter method is suitable for a single-problem analysis and a multiple-problem analysis should employ a nonparametric test [22]. Next, the experimental results are elaborated from two aspects, respectively.

6.1. Unconstrained Benchmark Function

The performances of these algorithms regarding twelve unconstrained benchmark functions were investigated at different scales, that is, 10, 100, and 1000. Table 8, Table 10 and Table 12 respectively summarize the descriptive statistical results at different scales. For 10-dimension benchmark functions, the superiority of the NSHS is not significant except for functions f₃ and f₄, as shown in Table 8. The test result shown in Table 9 indicates that the performance of the NSHS is significant better than the IHS at the 95% confidence level and the average error of the NSHS is less than other algorithms except for the DLHS. For 100-dimension benchmark functions, the average error of the NSHS is much less than other algorithms except for the SGHS, as shown in Table 11. Based on the data in Tables 12 and 13, we can see that the superiority of the proposed variant is extremely significant for 1000-dimension benchmark functions.

Table 8 

The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 10 dimensions.

Table 9 

Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 10 dimensions.

Table 10 

The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 100 dimensions.

Table 11 

Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 100 dimensions.

Table 12 

The descriptive statistical result of the relative errors of the six algorithms to solve twelve unconstrained benchmark functions with 1000 dimensions.

Table 13 

Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving twelve unconstrained benchmark functions with 1000 dimensions.

6.2. Constrained Benchmark Function

The performances of these algorithms with respect to the aforementioned optimization problems with constraints were investigated. The descriptive statistical result and the report on multiple comparisons are, respectively, shown in Tables 14 and 15. Clearly, the performance of the proposed variant is vastly superior to the HS, the IHS, the HSw, and the SGHS. Although the performance of the NSHS is slightly better than the DLHS, the evolutionary mechanism of the former is simpler and the NSHS does not require too much key-parameters setting. Moreover, the performance of the NSHS is significantly better than the DLHS for functions f₁₅ and f₁₈, seeing Table 14.

Table 14 

The descriptive statistical result of the relative errors of the six algorithms to solve six constrained benchmark functions.

Table 15 

Dunnett’s one-tailed (>) multiple comparisons between the NSHS and other five algorithms based on the samples of solving six constrained benchmark functions.

7. Conclusion

This paper presented a novel self-adaptive harmony search algorithm (NSHS) for optimization problems with continuous variables with the aim of improving the performance of the harmony search. To investigate the performance of the novel variant, many comparison experiments were conducted based on a test suite comprising six unimodal benchmark functions, six multimodal benchmark functions, and six constrained benchmark functions. Using nonparametric statistical tests, the NSHS’s robustness and superiority were detected. Experimental results on the effect of HMS and initial search ranges indicated that the performance of the NSHS did not depend on the initial HM. The results, when compared with related previous work, manifested that the NSHS performed dramatically better than other six algorithms for large-scale problems and constrained problems.

The originality of this work is mainly reflected in two aspects.(i)Proposed a simpler approach to the setting of key parameters based on musical practices. The proposed variant has only two key parameters HMCR and Bw, but they do not require the user’s inputs. Moreover, the parameters settings are in accordance with the features of the optimization problem to be solved, such as the scale and the boundaries. Moreover, this approach discards the PAR parameter and further simplifies the parameter setting. Musical improvisation is usually accompanied by fine-tuning; thus, it is senseless to consider this parameter and fine-tuning bandwidth at the same time.(ii)Developed a novel self-adaptive evolutionary scheme based on the standard deviation of the fitness function values in the memory to accelerate algorithm convergence and assure the reaching of a global optimum. This scheme automatically narrows the search space based on a suitable threshold and, meanwhile, solves the problem of the optimal balance between search diversity and intensity.

Acknowledgments

Many thanks to the anonymous reviewer for valuable comments that helped to improve this paper. This work was partly supported by the National Natural Science Foundation of China under Grant 71101003, 91224007, and 71332003 and the Fundamental Research Funds for the Central Universities under Grant YWF-13-D2-JC-12.