A New Filled Function Method with One Parameter for Global Optimization

Abstract

The filled function method is an effective approach to find the global minimizer of multidimensional multimodal functions. The conventional filled functions are numerically unstable due to exponential or logarithmic term and sensitive to parameters. In this paper, a new filled function with only one parameter is proposed, which is continuously differentiable and proved to satisfy all conditions of the filled function definition. Moreover, this filled function is not sensitive to parameter, and the overflow can not happen for this function. Based on these, a new filled function method is proposed, and it is numerically stable to the initial point and the parameter variable. The computer simulations indicate that the proposed filled function method is efficient and effective.

1. Introduction

More and more practical problems in science, economics, engineering, and other fields can be formulated as global optimization problems. Lots of researchers have been attracted to the field of global optimization. In recent years, many new theoretical and computational contributions have been reported for solving global optimization problems. Global optimization is mainly concerned with the characteristics and algorithms for the multimodal functions. In general, the existing approaches can be classified into two categories: deterministic methods ([1–16], e.g.) and probabilistic methods ([17–22], e.g.). The typical examples of the former are the filled function method (FFM) [3–9, 13–15], the trajectory method [2, 10], the tunneling method [12], and the covering method [1, 16], whereas ones of the latter are the clustering method [17], the methods reported in [18, 19], the simulated annealing method [22], and genetic algorithms [20, 21]. Furthermore, some hybrid deterministic and probabilistic algorithms ([23], e.g.) were proposed to solve practical problems.

However, the existence of multiple local minima of a general nonconvex objective function makes global optimization become a great challenge. For global optimization problems, there are two major issues.(1) How to find a lower minimizer of the objective function from a known local minimizer.(2) How to evaluate the convergence and, accordingly, design the stopping criteria.

In this paper, we will focus our research on the FFM and mainly issue (1). Among the existing methods for global optimization problems, the FFM appears to have several advantages over others mainly due to its relatively easy realization with a process that aims at successively finding smaller local minima. The FFM was firstly proposed by Ge in [3], which was used to solve the global minimizer of unconstrained multiextremum function. Later, many scholars have also done a lot of valuable work to improve this method ([5–9, 13–15, 24–28], e.g.). However, conventional filled functions are often nondifferentiable ([13, 25, 26], e.g.), need more than one adjustable parameter ([15], e.g.), or contain ill-conditioned terms ([3, 4, 9], e.g.). To overcome these shortcomings, some parameter-free filled functions ([13, 25], e.g.) and some filled functions without ill-conditioned terms ([26, 27], e.g.) are proposed; however, they are usually nondifferentiable, which often results in additional local minimizers. And some continuously differentiable filled functions with one parameter have been proposed ([8], e.g.), but the parameter is not easy to adjust. To deal with this problem, a new class of filled function with one parameter, which is continuously differentiable and the parameter is easy to adjust, is proposed in this paper. Based on this, a new filled function method with randomly and uniformly local search scheme is proposed, and the algorithm is numerical stability. In addition, the proposed method can be used to solve the multidimensional multi-modal problem.

2. Overview of the FFM

In this paper, we consider the following global optimization problem: min𝑓(𝑥)s.t.𝑥∈𝑅^𝑛, (P) where 𝑓(𝑥)∶𝑅^𝑛→𝑅 is a twice continuously differentiable function. Suppose 𝑓(𝑥) satisfies the condition 𝑓(𝑥)→+∞ as ‖𝑥‖→+∞. Then there exists a closed bounded domain Ω called an operating region that contains all local minimizers of 𝑓(𝑥). Then the global optimization Problem (P) can be rewritten into an equivalent form as follows: (𝑥)[]=min𝑓s.t.𝑥∈Ω=𝑙,𝑢𝑥∣𝑥∈𝑙≤𝑥≤𝑢,𝑙,𝑢∈𝑅^𝑛. (P1)

Because Ω can be estimated before Problem (P) is solved, so we can assume that Ω is known without loss of generality. We only consider Problem (P1) in the following.

2.1. Basic Concepts and Assumptions

In this paper, we adopt the following symbols: 𝑘: the iteration number; 𝑥^′_𝑘: the initial point in the 𝑘th iteration; 𝑥^∗_𝑘: the local minimizer of the objective function in the 𝑘th iteration; 𝑓^∗_𝑘: the function value at 𝑥^∗_𝑘; 𝐵^∗_𝑘: the basin of 𝑓(𝑥) at an isolated local minimizer 𝑥^∗_𝑘; 𝑥^∗: the global minimizer of the objective function.

Assumption 1. The function 𝑓(𝑥) in (P1) is continuously differentiable in 𝑅^𝑛, 𝑓(𝑥) has only a finite number of minimizers in Ω, and therefore every minimizer is an isolated minimizer.

The basin 𝐵^∗_𝑘 of 𝑓(𝑥) at a local minimizer 𝑥^∗_𝑘 is defined in [3, 4] as a connected domain, it contains 𝑥^∗_𝑘, and the steepest descent trajectory of 𝑓(𝑥) converges to 𝑥^∗_𝑘 from any initial point in 𝐵^∗_𝑘. The minimal radius of 𝐵^∗_𝑘 at an isolated minimizer 𝑥^∗_𝑘 is defined as 𝑅=inf_{𝑥∉𝐵∗𝑘}𝑥−𝑥^∗_𝑘. (1)

Radius 𝑅 is not zero if ∇²𝑓(𝑥^∗_𝑘) is positive definite. The basin of 𝑓(𝑥) at 𝑥^∗_𝑘 is said to be lower than another basin of 𝑓(𝑥) at another local minimizer 𝑥^∗₁ if and only if 𝑓(𝑥^∗_𝑘)<𝑓(𝑥^∗₁). The hill of 𝑓(𝑥) at 𝑥^∗_𝑘 is the basin of −𝑓(𝑥) at its isolated minimizer 𝑥^∗_𝑘.

The definition of the filled function was first proposed by Ge in [3] as follows.

Definition 2. Suppose 𝑥^∗_𝑘 is a current local minimizer of 𝑓(𝑥). 𝑃(𝑥,𝑥^∗_𝑘) is said to be a filled function of 𝑓(𝑥) at 𝑥^∗_𝑘, if it satisfies the following properties:(i)𝑥^∗_𝑘 is a strictly maximizer of 𝑃(𝑥,𝑥^∗_𝑘), and the whole basin 𝐵^∗_𝑘 of 𝑓(𝑥) at 𝑥^∗_𝑘 becomes a part of a hill of 𝑃(𝑥,𝑥^∗_𝑘);(ii)𝑃(𝑥,𝑥^∗_𝑘) has no minimizers or stable points in any basin of 𝑓(𝑥) higher than 𝐵^∗_𝑘;(iii) if 𝑓(𝑥) has a lower basin than 𝐵^∗_𝑘, then there is a point 𝑥^′_𝑘+1 in such a basin that minimizes 𝑃(𝑥,𝑥^∗_𝑘) on the line through 𝑥 and 𝑥^∗_𝑘.

Based on the filled functions, a global optimization problem can be solved via a two-phase cycle.

In Phase 1, we start from an initial point and use any local minimization method to find a local minimizer 𝑥^∗_𝑘 of 𝑓(𝑥).

In Phase 2, we construct a filled function at 𝑥^∗_𝑘 and minimize the filled function in order to identify a point 𝑥^′_𝑘+1 with 𝑓(𝑥^′_𝑘+1)<𝑓(𝑥^∗). If such a point 𝑥^′_𝑘+1 is found, 𝑥^′_𝑘+1 is certainly in a lower basin than 𝐵^∗_𝑘. We can then use 𝑥^′_𝑘+1 as an initial point in Phase 1 again, and hence we can find a better local minimizer 𝑥^∗_𝑘+1 of 𝑓(𝑥) with 𝑓(𝑥^∗_𝑘+1)<𝑓(𝑥^∗). This process repeats until no better solution can be found. The final local minimum will be a global minimizer of 𝑓(𝑥).

2.2. Overview of the FFM

As a deterministic yet a universal global optimization technique, the development of the FFM undergoes the following generations. The representative examples of the FFM in the first generation are 𝑃-functions [3] and 𝐺-functions [4] given by 𝑃𝑥,𝑟,𝜌=exp−‖𝑥−𝑥^∗_𝑘‖²/𝜌²(𝑥,𝐺𝜌𝑟+𝑓)𝑥,𝑟,𝜌=−²ln𝑟+𝑓(𝑥)+𝑥−𝑥^∗_𝑘². (2)

The first-generation filled functions share a common feature: there are two adjustable parameters, 𝑟 and 𝜌, which greatly affect the performance of the algorithms and need to be appropriately adjusted. However, how to adjust the parameters is a very difficult task. Due to these drawbacks, the second-generation filled functions were proposed, and they have only one parameter. Among them, the representative examples are 𝑄-function [4] and 𝑅-function [29]: 𝑄(𝑥,𝑎)=−𝑓(𝑥)−𝑓𝑥^∗_𝑘exp𝑎𝑥−𝑥^∗_𝑘², (3) where the adjustable parameter is 𝑎.

Consider 𝑅()=1𝑥,𝑎1+𝑥−𝑥^∗_𝑘⋅𝜑_𝑞𝑓(𝑥)−𝑓𝑥^∗_𝑘+𝑞, (4) where 𝜑_𝑞(𝑡)=−𝑞exp³𝑡,𝑡≠0,0,𝑡=0,0<𝑞<min_{𝑥∗𝑘(𝑃)(𝑃)∈𝐿⧵𝐺}𝑓𝑥^∗_𝑘−𝑓𝑥^∗_𝑔, (5) and 𝑥^∗_𝑔∈𝐺(𝑃); 𝐿(𝑃) and 𝐺(𝑃) stand for the set of local minimizers of 𝑓(𝑥) and global minimizers of 𝑓(𝑥), respectively.

These filled functions only have one parameter than those in the first generation. However, these functions are liable to be ill-conditioned in practice since their function values increase exponentially due to an exponential function. As the adjustable parameter becomes larger and larger, which is required by the FFM itself, the rapidly increasing exponential function value may result in an overflow in the computation. To overcome this drawback, 𝐻-function was proposed by Liu [27]: 𝐻()=1𝑥,𝑎ln1+𝑓(𝑥)+𝑓𝑥^∗_𝑘−𝑎𝑥−𝑥^∗_𝑘². (6)

The 𝐻-function retains the advantage of the 𝑄-function with only a single parameter and without exponential terms. The performance of the 𝐻-function in numerical experiments for a large set of testing functions was quite satisfactory [27]. 𝐻(𝑥,𝑎) can be regarded as the third generation due to the absence of the exponential term. Nevertheless, the 𝐻-function has a drawback which is discontinuous at 𝑥∈𝑆={𝑥∣𝑓(𝑥)=𝑓(𝑥^∗_𝑘)}.

However, on the one hand, the continuity and differentiability on the FFM are required for the convergence analysis in theory [15]. On the other hand, computationally, most local minimization algorithms for the numerical nonlinear programming require the gradient information in their procedures (readers can refer to [30] or [31] for detail). Thus, it is very necessary to develop a continuously differentiable filled function with as few parameters as possible. There was already some work in this area ([8], e.g.), but the parameters of the filled functions are not easy to adjust. Based on this consideration, a continuously differentiable filled function with one parameter is designed, and the parameter, which can be as large as possible, is relatively easy to adjust.

3. A New Filled Function and Its Properties

Definition 2 relies on the concept of the basin and hill of 𝑓(𝑥), which requires that the minima in the operating region are isolated, and Definition 2 also requires that there exists a minimal point of 𝑓(𝑥) along a line. This is more difficult to be guaranteed. Therefore, many improvements in the definition are given in the literatures ([6, 14, 28], e.g.), which make it more convenient to construct a new filled function. In this section, we use the definition in [28] for Problem (P1).

Definition 3. Suppose 𝑥^∗_𝑘 is a current local minimizer of 𝑓(𝑥). 𝑃(𝑥,𝑥^∗_𝑘) is said to be a filled function of 𝑓(𝑥) at 𝑥^∗_𝑘, if it satisfies the following properties:(i)𝑥^∗_𝑘 is a strictly maximizer of 𝑃(𝑥,𝑥^∗_𝑘);(ii) for any 𝑥∈Ω₁, one has 0∉∇𝑃(𝑥,𝑥^∗_𝑘), and Ω₁={𝑥∈Ω/𝑓(𝑥)≥𝑓(𝑥^∗_𝑘),𝑥≠𝑥^∗_𝑘};(iii) if Ω₂={𝑥/𝑓(𝑥)<𝑓(𝑥^∗_𝑘),𝑥∈Ω} is not empty, then there exists a point 𝑥^′_𝑘∈Ω₂ such that 𝑥^′_𝑘 is a local minimizers of 𝑃(𝑥,𝑥^∗_𝑘).

Note that Definition 3 about the filled function is different from the definition mentioned in [3]. It is much easier to construct a new filled function by Definition 3, and the local optimal solution of the filled function can be easily found. For example, suppose that 𝑥^∗_𝑘 is not a global minimizer, then, by condition (iii) of Definition 3, we can find a point 𝑥^′_𝑘∈Ω₂ by minimizing 𝑃(𝑥,𝑥^∗_𝑘). Therefore, we can obtain a lower local minimizer 𝑥^∗_𝑘+1 of 𝑓(𝑥) by searching 𝑓(𝑥) starting at 𝑥^′_𝑘 via local search schemes. In the process of minimizing 𝑃(𝑥,𝑥^∗_𝑘), it does not require, unlike Definition 2, that 𝑥^∗_𝑘+1 must be on the straight line through 𝑥^′_𝑘 and 𝑥^∗_𝑘. So the design of the filled function is much easier and flexible.

In order to find a global minimizer of 𝑓(𝑥), the major issue of the filled function method is to find a lower minimizer of 𝑓(𝑥) or justify whether the obtained local minimizer is a global minimizer of 𝑓(𝑥). This heavily relies on the performance of the filled function used.

In this section, we propose a new filled function for Problem (P1) at a local minimizer 𝑥^∗_𝑘 as follows: 𝑃𝑥,𝑥^∗_𝑘=−𝑥−𝑥^∗_𝑘²𝑔𝑓(𝑥)−𝑓𝑥^∗_𝑘,𝑔(𝑡)=𝜋2𝑡,𝑡≥0,𝑟⋅arctan²+𝜋2,𝑡<0, (7) where 𝑟 is an adjustable positive real number as large as possible, used as the weight factor.

Note that the proposed filled function has some advantages: first, it has one parameter 𝑟 which is a positive real number as large as possible, thus it is easy to adjust, and, second, it is continuously differentiable, which makes it more easily solved by the existing local optimization method; finally, arctan(𝑡²)∈[0,𝜋/2) is bounded, which ensures that the calculation of 𝑃(𝑥,𝑥^∗_𝑘) will not overflow and is of numerical stability. The following theorems show that 𝑃(𝑥,𝑥^∗_𝑘) satisfies Definition 3.

Theorem 4. Suppose that 𝑥^∗_𝑘 is a local minimizer of 𝑓(𝑥), then 𝑥^∗_𝑘 is a strictly local maximizer of 𝑃(𝑥,𝑥^∗_𝑘).

Proof. Since 𝑥^∗_𝑘 is a local minimizer of 𝑓(𝑥), then there exists a small positive real number 𝜀, and a neighborhood 𝛿=𝑈(𝑥^∗_𝑘,𝜀), such that, for all 𝑥∈𝛿, when 𝑥≠𝑥^∗_𝑘, then 𝑓(𝑥)>𝑓(𝑥^∗_𝑘), and 𝑃(𝑥,𝑥^∗_𝑘)=−𝜋/2‖𝑥−𝑥^∗_𝑘‖²<0=𝑃(𝑥^∗_𝑘,𝑥^∗_𝑘). Thus, 𝑥^∗_𝑘 is a strictly local maximizer of 𝑃(𝑥,𝑥^∗_𝑘).

Theorem 4 clearly shows that 𝑃(𝑥,𝑥^∗_𝑘) satisfies the property (i) of Definition 3.

Theorem 5. Suppose Assumption 1 is satisfied, 𝑥^∗_𝑘 is a local minimizer of 𝑓(𝑥), and for any 𝑥∈Ω₁={𝑥/𝑓(𝑥)≥𝑓(𝑥^∗_𝑘),𝑥∈Ω,𝑥≠𝑥^∗_𝑘}, one has 0∉∇𝑃(𝑥,𝑥^∗_𝑘).

Proof. For any 𝑥∈Ω₁, 𝑓(𝑥)≥𝑓(𝑥^∗_𝑘), and for 𝑥≠𝑥^∗_𝑘, one has ∇𝑃(𝑥,𝑥^∗_𝑘)=−𝜋(𝑥−𝑥^∗_𝑘)≠0. Consequently, 0∉∇𝑃(𝑥,𝑥^∗_𝑘).

Theorem 6. Suppose Assumption 1 is satisfied, 𝑥^∗_𝑘 is a local minimizer of 𝑓(𝑥), and Ω₂={𝑥/𝑓(𝑥)<𝑓(𝑥^∗_𝑘),𝑥∈Ω} is not empty; then there exists a point 𝑥^′_𝑘∈Ω₂, such that 𝑥^′_𝑘 is a local minimizer of 𝑃(𝑥,𝑥^∗_𝑘).

Proof. Let Ω₃={𝑥/𝑓(𝑥)≤𝑓(𝑥^∗_𝑘),𝑥∈Ω}, 𝜕Ω₂={𝑥/𝑓(𝑥)=𝑓(𝑥^∗_𝑘),𝑥∈Ω}; then Ω₃=Ω₂∪𝜕Ω₂, where 𝜕Ω₂ is the boundary of Ω₂ (and Ω₃). Since 𝑓(𝑥) is continuous and both Ω₃ and 𝜕Ω₂ are contained in Ω, then both Ω₃ and 𝜕Ω₂ are bounded closed sets. For any 𝑥∈𝜕Ω₂, 𝑃𝑥,𝑥^∗_𝑘𝜋=−2⋅𝑥−𝑥^∗_𝑘². (8) Since 𝜕Ω₂ is bounded closed set, then there exists 𝑥∈𝜕Ω₂, such that min_{𝑥∈𝜕Ω2}𝑃𝑥,𝑥^∗_𝑘=𝑃𝑥,𝑥^∗_𝑘𝜋=−2⋅𝑥−𝑥^∗_𝑘². (9) Note that, for any 𝑥∈Ω₂, 𝑓(𝑥)<𝑓(𝑥^∗_𝑘), and 𝑃𝑥,𝑥^∗_𝑘=−𝑥−𝑥^∗_𝑘²×𝜋2+𝑟⋅arctan𝑓(𝑥)−𝑓𝑥^∗_𝑘². (10) Since Ω₂ is not empty, there exists a point 𝑥₁∈Ω₂=intΩ₃ such that 𝑃𝑥₁,𝑥^∗_𝑘=−𝑥₁−𝑥^∗_𝑘²×𝜋2+𝑟⋅arctan𝑓𝑥₁−𝑓𝑥^∗_𝑘², (11) when 𝜋𝑟>2⋅arctan𝑓𝑥₁−𝑓𝑥^∗_𝑘²𝑥−𝑥^∗_𝑘²𝑥₁−𝑥^∗_𝑘²−1≜𝑟, (12) for example, 𝑟=𝑟+1; one has 𝑃(𝑥₁,𝑥^∗_𝑘)<𝑃(𝑥,𝑥^∗_𝑘). In other words, when 𝑟=𝑟+1, at least there exists a point 𝑥₁∈intΩ₃, such that 𝑃(𝑥₁,𝑥^∗_𝑘)<𝑃(𝑥,𝑥^∗_𝑘). Since 𝑃(𝑥,𝑥^∗_𝑘) is continuous on closed bounded set Ω₃, 𝑃(𝑥,𝑥^∗_𝑘) must have a global minimizer 𝑥^′_𝑘 on Ω₃ for this fixed 𝑟. Note that 𝑃(𝑥^′_𝑘,𝑥^∗_𝑘)≤𝑃(𝑥₁,𝑥^∗_𝑘)<𝑃(𝑥,𝑥^∗_𝑘), and 𝑥 is a global minimizer of 𝑃(𝑥,𝑥^∗_𝑘) on 𝜕Ω₃ (and 𝜕Ω₂), then 𝑥^′_𝑘∉𝜕Ω₃, and thus 𝑥^′_𝑘∈intΩ₃=Ω₂, that is, 𝑥^′_𝑘 is also a global minimizer of 𝑃(𝑥,𝑥^∗_𝑘) on Ω₂; The proof is completed.

Theorems 4–6 state that the proposed filled function satisfies the conditions of Definition 3.

4. Filled Function Algorithm

4.1. A New Local Search Method: RULS

Conventional local optimization method minimizes function 𝑓(𝑥) directly from the initial point 𝑥^′_𝑘, and then a local minimizer 𝑥^∗_𝑘 is obtained. In the process, the function is called repeatedly when the derivative and the search direction are calculated; in addition, the number of function evaluations is also increased in the one-dimensional search process. More importantly, with the increase of dimension, the number of iterations and the number of function evaluations are also increasing. Thus, the amount of computation for the traditional local search methods is very large, which would affect the computational efficiency. In this subsection, a new local search strategy called randomly and uniformly local search (RULS) is given, which can make the initial point closer to the local optimal solution, thereby the number of iterations and the number of function evaluations will be reduced, and the convergence speed will be accelerated.

Algorithm RULS. The details for RULS are as follows.

Step 1. 𝑥^′_𝑘∈Ω=[𝑙,𝑢]={𝑥/𝑙≤𝑥≤𝑢,𝑙,𝑢∈𝑅^𝑛} is an initial point for the 𝑘th iteration, and 𝑀 is the set of all points that have been used so far. Calculate 𝑎=(∑^|𝑀|_𝑖=1‖𝑥^′_𝑘−𝑥_𝑖‖)/|𝑀|, 𝑥_𝑖∈𝑀, where |𝑀| denotes the elements number of the set 𝑀.

Step 2. Γ∈Ω is a neighborhood at 𝑥^′_𝑘, where Γ=[Γ_𝑙,Γ_𝑢]=[𝑥^′_𝑘−𝑎/𝑚⋅𝑒,𝑥^′_𝑘+𝑎/𝑚⋅𝑒], 𝑚>1, and 𝑒=(1,…,1)^𝑇∈𝑅^𝑛. Set Γ_𝑙=𝑥^′_𝑘−𝑎/𝑚⋅𝑒, and Γ_𝑢=𝑥^′_𝑘+𝑎/𝑚⋅𝑒. For 𝑖=1 to 𝑛.  If Γ_𝑙(𝑖)<𝑙(𝑖) then   Γ_𝑙(𝑖)=𝑙(𝑖)  Endif   If Γ_𝑢(𝑖)>𝑢(𝑖) then   Γ_𝑢(𝑖)=𝑢(𝑖)  Endif Endfor

Randomly and uniformly select 𝑏∗𝑛 points 𝑌_𝑘={𝑦₁,𝑦₂,…,𝑦_𝑏𝑛} in Γ, where 𝑛 is the dimension of the problem, and 𝑏 is the number of points selected in each dimension, 𝑀={𝑀,𝑌_𝑘}.

Step 3. Calculate the function value of these points 𝑌_𝑘={𝑦₁,𝑦₂,…,𝑦_𝑏𝑛} to find the points 𝑌 corresponding to the smallest function value, where 𝑌=argmin_{𝑦𝑖∈𝑌𝑘,𝑖∈{1,2,…,𝑏𝑛}}𝑓𝑦₁,𝑓𝑦₂,…,𝑓𝑦_𝑏𝑛. (13)

Let 𝑋₁=argmin_{𝑥′𝑘,𝑦∈𝑌∪𝑥𝑘}{𝑓(𝑥^′_𝑘),𝑓(𝑦)} as a set of the initial points.

Step 4. Starting from any point 𝑧_𝑘∈𝑋₁, minimize 𝑓(𝑥) by using a local optimization method to obtain a local minimizer sequence 𝑋₂={𝑧^∗_𝑘} and a local minimum value sequence 𝐹₂={𝑓(𝑧^∗_𝑘)}, set 𝑀={𝑀,𝑋₂}.

Step 5. Calculate 𝑓(𝑥^∗_𝑘)=min{𝑓(𝑧^∗_𝑘)} to obtain a local minimizer 𝑥^∗_𝑘, 𝑀={𝑀,𝑥^∗_𝑘}.

Explanation of Algorithm RULS. uniformly and randomly generate some points near the initial point 𝑥^′_𝑘, select one of the points corresponding to the smallest function value as the new initial point to minimize the function 𝑓(𝑥), and obtain a local minimizer 𝑥^∗_𝑘. Diagram is shown in Figures 4(d₁), 4(d₂), and 4(d₃).

4.2. A New Filled Function Algorithm (NFFA)

Based on the results of the previous content, a new filled function algorithm is proposed as follows.

Algorithm 1 (NFFA)

Step 1. (initialization step)(a) Choose a tolerance 𝜀>0, for example, 𝜀∶=1.0𝑒−3.(b) Choose a large integer constant 𝐵, a positive real number as large as possible 𝑟, and a small constant 𝛿∶=1.0𝑒−3.(c) Choose a positive integer 𝐶<𝐵.(d) Set 𝑓(𝑥^∗_𝑘−1)=+∞, and 𝑘∶=1.

Step 2. Randomly and uniformly select 𝑏∗𝑛 points 𝑌_𝑘 in the operating region, where 𝑌_𝑘={𝑦₁,𝑦₂,…,𝑦_𝑏𝑛}, 𝑦_𝑖∈Ω, 𝑖∈{1,2,…,𝑏𝑛}, 𝑛 is the dimension of the problem, and 𝑏 is the number of points selected in each dimension. Calculate the function value of these points, and find the points corresponding to the smallest function value 𝑌=argmin_{𝑦𝑖∈𝑌𝑘,𝑖∈{1,2,…,𝑏𝑛}}{𝑓(𝑦₁),𝑓(𝑦₂),…,𝑓(𝑦_𝑏𝑛)}. Select for all 𝑥_𝑘∈𝑌.

Step 3. Starting from 𝑥_𝑘, minimize 𝑓(𝑥) by using RULS to obtain a local minimizer 𝑥^∗_𝑘; then go to Step 6.

Step 4. Construct a filled Function (7) at the local minimizer 𝑥^∗_𝑘, and Go to Step 5.

Step 5. Set 𝑥=𝑥^∗_𝑘+𝛿𝑒^𝑖, use 𝑥 as the initial point to minimize 𝑃(𝑥,𝑥^∗_𝑘) by using RULS, and find minimizers 𝑥^′ of 𝑃(𝑥,𝑥^∗_𝑘). Let 𝑥_𝑘+1=𝑥^′ and 𝑘=𝑘+1, and then go to Step 3.

Step 6 (termination step). If 𝑘>𝐶 and |𝑓(𝑥^∗_𝑘)−𝑓(𝑥^∗_𝑘−𝐶)|≤𝜀, (𝑘=1,2,…) or 𝑘>𝐵, the algorithm will stop, and 𝑥^∗=𝑥^∗_𝑘 is taken as a global minimizer of 𝑓(𝑥); otherwise, increase 𝑟, and go to Step 4.

Some Explanations about the above Filled Function Algorithm(1) In minimization to 𝑓(𝑥) and 𝑃(𝑥,𝑥^∗_𝑘), a local optimization method has to be selected firstly. In our algorithm, the MATLAB function “fmincon” is used.(2)In Step 5, the smaller 𝛿 has to be chosen accurately, in the algorithm; the 𝛿 is selected to guarantee that ‖∇𝑃(𝑥,𝑥^∗_𝑘)‖ is greater than a threshold.(3) Step 5 means that if local minimizer 𝑥^′ of 𝑃(𝑥,𝑥^∗_𝑘) is found in Ω with 𝑓(𝑥^′)<𝑓(𝑥^∗_𝑘), we can use 𝑥^′ as the initial point to minimize 𝑓(𝑥) and obtain a better local minimizer of 𝑓(𝑥).(4) In Step 6, it is necessary to increase 𝑟, because the lower local minimizer may not be found when 𝑟 is not large enough.

5. Numerical Experiments

5.1. Test Problems

In this section, the proposed algorithm is tested on Problem 1 and Problems 2–9 taken from [14].

Problem 1 (one-dimensional function). We have min𝑓(𝑥)=sin𝑥+sin2𝑥−cos4𝑥,s.t.−2≤𝑥≤4. (14) The global minimizer is 𝑥^∗=−1.4523, and the global optimal value is 𝑓(𝑥^∗)=−2.1175.

Problem 2 (two-dimensional function). We have min𝑓(𝑥)=1−2𝑥₂+𝑐sin4𝜋𝑥₂−𝑥₁²+𝑥₂−0.5sin2𝜋𝑥₁²,s.t.0≤𝑥₁≤10,−10≤𝑥₁≤0, (15) where 𝑐=0.2,0.5, and 0.05. The global minimum value is 𝑓(𝑥^∗)=0 for all 𝑐.

Problem 3 (three-hump back camel function). We have min𝑓(𝑥)=2𝑥²₁−1.05𝑥⁴₁+16⋅𝑥⁶₁−𝑥₁𝑥₂+𝑥²₂,s.t.−3≤𝑥₁≤3,−3≤𝑥₁≤3. (16) The global minimizer is 𝑥^∗=(0,0)^𝑇, and the global optimal value is 𝑓(𝑥^∗)=0.

Problem 4 (six-hump back camel function). We have min𝑓(𝑥)=4𝑥²₁−2.1𝑥⁴₁+13⋅𝑥⁶₁−𝑥₁𝑥₂−4𝑥²₂+4𝑥⁴₂,s.t.−3≤𝑥₁≤3,−3≤𝑥₁≤3. (17) The global minimizers are 𝑥^∗=(−0.0898,−0.7127)^𝑇 and 𝑥^∗=(0.0898,0.7127)^𝑇, and the global optimal value is 𝑓(𝑥^∗)=−1.0316.

Problem 5 (Treccani function). We have min𝑓(𝑥)=𝑥⁴₁+4𝑥³₁+4𝑥²₁+𝑥²₂,s.t.−3≤𝑥₁≤3,−3≤𝑥₁≤3. (18) This problem has two global minimizers in total, which are 𝑥^∗=(0,0)^𝑇 and 𝑥^∗=(−2,0)^𝑇, and the global optimal value is 𝑓(𝑥^∗)=0.

Problem 6 (Goldstein and Price function). We have min𝑓(𝑥)=𝑔(𝑥)ℎ(𝑥),s.t.−3≤𝑥₁≤3,−3≤𝑥₁≤3, (19) where 𝑔(𝑥)=1+(𝑥₁+𝑥₂+1)²(19−14𝑥₁+3𝑥²₁−14𝑥₂+6𝑥₁𝑥₂+3𝑥²₂), and ℎ(𝑥)=30+(2𝑥₁−3𝑥₂)²(18−32𝑥₁+12𝑥²₁+48𝑥₂−36𝑥₁𝑥₂+27𝑥²₂).

The global minimizer is 𝑥^∗=(0,−1)^𝑇 and the global optimal value is 𝑓(𝑥^∗)=3.

Problem 7 (two-dimensional Shubert function). We have min𝑓(𝑥)=⁵_𝑖=1𝑖cos(𝑖+1)𝑥₁×+𝑖_5𝑖=1𝑖cos(𝑖+1)𝑥₂+𝑖,s.t.0≤𝑥₁≤10,0≤𝑥₁≤10. (20) This problem has two global minimizers in total, which are 𝑥^∗=(4.8581,5.4829)^𝑇 and 𝑥^∗=(5.4829,4.8581)^𝑇. The global minimum value is 𝑓(𝑥^∗)=−186.7309.

Problem 8 (Shekel’s function). We have min𝑓(𝑥)=−⁵_𝑖=1⁴_𝑗=1𝑥_𝑗−𝑎_𝑖𝑗²+𝑐_𝑖⁻¹,s.t.0≤𝑥_𝑗≤10,𝑗=1,2,3,4, (21) where the coefficients 𝑎_𝑖𝑗, 𝑖=1,2,3,4, and 5, 𝑗=1,2,3, and 4, are given in Table 1. The global minimizer is 𝑥=(4.0000,4.0001,4.000,4.0001)^𝑇 and the global optimal value is 𝑓(𝑥^∗)=−10.1529.

Table 1 

The coefficients 𝑎_𝑖𝑗,𝑐_𝑖,𝑖=1,2,3,4,5,𝑗=1,2,3,4.

Problem 9 (𝑛-dimensional function). We have (𝑥)=𝜋min𝑓𝑛⋅10sin²𝜋𝑥₁+𝑔(𝑥)+𝑥_𝑛−1²,s.t.−10≤𝑥_𝑖≤10,𝑖=1,2,…,𝑛, (22) where 𝑔(𝑥)=∑^𝑛−1_𝑖=1[(𝑥_𝑖−1)²(1+10sin²𝜋𝑥_𝑖+1)]. The global minimizer of this problem is 𝑥^∗=(1,…,1)^𝑇 for all 𝑛.

5.2. The Experimental Results

In the following, the procedure for solving Problem 1 is given, respectively, by using the traditional filled function method and the proposed algorithm, and a comparison of these two algorithms is also given.

5.2.1. Conventional Filled Function Method for Solving Problem 1

The procedure of the conventional filled function method is as follows.(1) Randomly select point 𝑥₁=1.043 in the operating region Ω; then minimize 𝑓(𝑥) from 𝑥₁ by using local minimization function “fmincon” to obtain a local minimizer 𝑥^∗₁=1.71.(2) Construct a filled Function (7) at 𝑥^∗₁=1.71, where 𝑓(𝑥)𝑔(𝑡)=𝜋=sin𝑥+sin2𝑥−cos4𝑥,2𝑡,𝑡≥0,5⋅arctan²+𝜋2,𝑡<0. (23) (3) Minimize 𝑃(𝑥,𝑥^∗₁) to obtain a local minimizer 𝑥₂=−0.32 of 𝑃(𝑥,𝑥^∗₁); the specific processes of Steps (1), (2), and (3) are shown in Figure 1. Note that Figure 1(a′) is to enlarge the rectangular box in Figure 1(a), behind Figures 2(b′) and 3(c′), and so on, also said this, therefore, not to repeat explained.(4) Minimize 𝑓(𝑥) from 𝑥₂=−0.32 to obtain a local minimizer 𝑥^∗₂=−0.2 of 𝑓(𝑥). Construct a filled function 𝑃(𝑥,𝑥^∗₂) at 𝑥^∗₂, where 𝑟=5, and then minimize the filled function 𝑃(𝑥,𝑥^∗₂) to obtain a local minimizer 𝑥₃=−1.15 of 𝑃(𝑥,𝑥^∗₂) (as shown in Figure 2).(5) Minimize 𝑓(𝑥) from 𝑥₃=−1.15 to obtain a local minimizer 𝑥^∗₃=−1.4523. Construct a filled function 𝑃(𝑥,𝑥^∗₃) at 𝑥^∗₃, where 𝑟=5, which is shown in Figure 3, and then minimize 𝑃(𝑥,𝑥^∗₃). Without any stable point is found. Then the iteration terminates, and set 𝑥^∗=𝑥^∗₃. The global minimizer 𝑥^∗=−1.4523 and the global minimum value 𝑓(𝑥^∗)=−2.1175 are obtained.

Figure 1 

((a) and (a′)) The process of Steps (1), (2), and (3) in Section 5.2.1.

Figure 2 

((b) and (b′)) The process of Step (4) in Section 5.2.1.

Figure 3 

((c) and (c′)) The process of Step (5) in Section 5.2.1.

Figure 4 

((d1), (d2), and (d3)) The process of Steps (1), (2), and (3) in Section 5.2.2.

Conventional filled function algorithm is run independently 50 times; the mean number of function evaluations is 18.7119.

5.2.2. The Proposed Filled Function Algorithm for Solving Problem 1

The procedure of the proposed filled function algorithm for solving Problem 1 is as follows.(1) Uniformly and randomly generate three points 𝑥₀₁=0, 𝑥₀₂=2.2, and 𝑥₀₃=3.35 in the operating region, set 𝑀={𝑥₀₁,𝑥₀₂,𝑥₀₃}, and calculate the function value 𝑓(𝑥) of the three points. Find the point 𝑥^′₀=0 corresponding to the smallest function value, as shown in (d₁) of Figure 4.(2) Calculate 𝑎=∑^|𝑀|_𝑖=1‖𝑥^′₀−𝑥_𝑖‖/|𝑀|=(‖0−2.2‖+‖0−3.35‖)/2=2.775 by using the rule in RULS, where 𝑥_𝑖∈𝑀, 𝑖∈{1,2,…,|𝑀|}; let 𝑚=5, and then Γ=𝑥^′₀−𝑎𝑚,𝑥^′₀+𝑎𝑚=𝑥^′₀−0.555,𝑥^′₀+0.555. (24) Randomly select three points 𝑌_𝑘 in Γ, calculate their value of 𝑓(𝑥) to find the point 𝑥₁ corresponding to the smallest function value, as shown in (d₂) of Figure 4, and set 𝑀={𝑀,𝑥₁}.(3) Minimize 𝑓(𝑥) from 𝑥₁ by using a local optimization method to find a local minimizer 𝑥^∗₁=−0.2 and a minima 𝑓(𝑥^∗₁)=−1.285, as shown in (d₃) of Figure 4, and set 𝑀={𝑀,𝑥^∗₁}. Steps (2) and (3) describe the specific process of RULS, as shown in (d₂) and (d₃) of Figure 4.(4) Construct a filled Function (7) at 𝑥^∗₁=−0.2, as shown in Figure 5, where 𝑓(𝑥)=sin𝑥+sin2𝑥−cos4𝑥, 𝑔(𝑡)=𝜋2𝑡,𝑡≥0,4⋅arctan²+𝜋2,𝑡<0. (25)(5) Optimize the function 𝑃(𝑥,𝑥^∗₁) at 𝑥^∗₁=−0.2 by using RULS method to obtain a minimizer 𝑥₂=−1.53 of 𝑃(𝑥,𝑥^∗₁), as shown in (d₄) of Figure 4. Set 𝑥^′₀=𝑥₂ and 𝑀={𝑀,𝑥^∗₂}, and then minimize 𝑓(𝑥) by using RULS to obtain 𝑥^∗₂=−1.4523 and 𝑓(𝑥^∗₂)=−2.1175.(6) Construct a filled function 𝑃(𝑥,𝑥^∗₂) at 𝑥^∗₂, where 𝑟=4, as shown in Figure 6.(7) Function 𝑃(𝑥,𝑥^∗₂) without a stable point in {𝑥∈Ω∣𝑥≠𝑥^∗₂}, so the iteration terminates. Set 𝑥^∗=𝑥^∗₂. The global minimizer 𝑥^∗=−1.4523 and the global minimum value 𝑓(𝑥^∗)=−2.1175 are obtained.

Figure 5 

((d4) and (d′4)) The process of Step (4) in Section 5.2.2.

Figure 6 

((d5) and (d′5)) The process of Step (5) in Section 5.2.2.

The proposed algorithm runs independently 50 times; the mean number of function evaluations is 13.6168, which is significantly less than the former result that uses conventional filled function method.

Both experimental results of Sections 5.2.1 and 5.2.2 and the mean number of function evaluations show that the proposed filled function method is more efficient.

5.2.3. The Proposed Filled Function Method for Solving the Problems Taken from [14]

The proposed algorithm is executed on the above test Problems 2–9. The results obtained by the proposed algorithm and the comparison with [14] are listed in Tables 2, 3, and 4.

Table 2 

The results obtained by the proposed algorithm.

Table 3 

The results obtained by the proposed algorithm and the algorithm in [14].

Table 4 

The results obtained by the proposed algorithm for solving Problem 9 with different dimensions.

The symbols used in Tables 2, 3, and 4 are given as follows: No.: the number of the test problems; 𝑛: the dimension of the test problems; 𝑟: the parameter of the filled function; 𝑘: the iteration number; 𝑞: the total number of function and gradient evaluations of 𝑓(𝑥) and 𝑃(𝑥,𝑥^∗_𝑘); 𝑥^∗_𝑘: the local minimizer of the objective function in the 𝑘th iteration; 𝑓^∗_𝑘: the function value at 𝑥^∗_𝑘; 𝑥^∗: the global minimizer of the objective function; times: the once running time (second); 𝑓-mean: the mean function value, respectively, in the 50 runs; 𝑓-best: the best function value, respectively, in the 50 runs; 𝑓-std: the standard deviation of function value, respectively, in the 50 runs; ratio: the ratio of the successful runs, that is, the ratio of the runs finding the true optimal solution in the 50 runs.

It can be seen from Tables 2, 3, and 4 that the proposed algorithm has the following advantages.

(1) More Minimizers Are Obtained. In the proposed algorithm, the initial points are randomly and uniformly generated. This is helpful to get more different global minimizers in the 50 runs. For example, for Problems 2, 5, and 7, more optimal solutions are obtained by the proposed algorithm than those obtained by the algorithm in the literature [14].

(2) More Effective and Efficient. The optimal solutions of all test problems can be found by the proposed algorithm. This indicates the effectiveness of the proposed algorithm. In addition, it can be seen from Table 3 that both the mean number of function evaluations 𝑞 and the once running time obtained by the proposed algorithm are smaller than that obtained by the algorithm in [14]. This also indicates the efficiency of the proposed algorithm.

(3) More Stability. It can be seen from column ratio in Table 2 that the ratio of the successful runs is more than 95.92%, and most of them are 100%, which shows that the proposed algorithm is stable. In addition, it can be seen from columns 𝑓-mean and 𝑓-std that difference between 𝑓-mean and 𝑓-best is small, and 𝑓-std is also small. This also indicates that the proposed algorithm is stable and robust to the initial points and parameter variation.

(4) Applicable to Multidimensional Problems. In Table 4, Problem 9 with different dimensions is tested, and the numerical results indicate that the proposed algorithm is suitable for solving multidimensional problems.

6. Conclusions

The filled function method is an approach to find the global minima of multi-modal functions. The existing filled functions have some drawbacks such as being nondifferentiable at some point in search domain, including the exponential term, containing more than one adjustable parameter, and being discontinuous. In this paper, a filled function with one parameter is designed, which is continuously differentiable, and it can overcome the existing shortcomings. Based on this, a new filled function method with randomly and uniformly local search scheme is proposed, and the algorithm is numerically stable. The results of the computer simulations indicate that the proposed filled function method is effective and efficient.

Acknowledgment

The paper is supported by The National Natural Science Foundation of China (no. 61272119).