Finding the Roots of System of Nonlinear Equations by a Novel Filled Function Method

Abstract

We present a novel filled function approach to solve box-constrained system of nonlinear equations. The system is first transformed into an equivalent nonsmooth global minimization problem, and then a new filled function method is proposed to solve this global optimization problem. Numerical experiments on several test problems are conducted and the computational results are also reported.

1. Introduction

Consider the system of nonlinear equations of the form: (SNE)∶𝐹(𝑥)=0,𝑥∈𝑋, where the mapping𝐹∶𝑅^𝑛→𝑅^𝑚 is continuous, and 𝑋⊂𝑅^𝑛 is a box set.

Denote 𝐹(𝑥)=(𝑓₁(𝑥),𝑓₂(𝑥),…,𝑓_𝑚(𝑥))^𝑇, and let 𝑓(𝑥)=Σ^𝑚_𝑘=1|𝑓_𝑘(𝑥)|. To obtain the roots of (SNE), we may solve the following global optimization problem: (𝑃)∶min_𝑥∈𝑋𝑓(𝑥). Obviously, if (SNE), has at least one root, then the global optimizers of the problem (𝑃) with the zero function value correspond to the roots of (SNE).

There are many theories and algorithms devoted to global optimization. Among of them, filled function method is a particularly useful tool (see [1–7]). The filled function method is originally introduced to solve smooth global optimization problem. In this paper, it is extended to solve nonsmooth global optimization problem (𝑃). The filled function method for nonsmooth global optimization contains two phases: local minimization and filling. The local minimization phase is intended to search for a local optimizer 𝑥^∗ of the problem (𝑃), and the filling phase aims to identify an improved initial point by minimizing the following filled function constructed at 𝑥^∗∶(FFP)min_𝑥∈𝑋𝐹(𝑥,𝑥^∗). The two phases are performed alternatively until no better optimizer can be found, and then the current optimizer is regarded as a global minimizer.

Comparing with the filled functions proposed in [1–7], our filled function has the following features.(1)Since the functions 𝑓_𝑖(𝑖=1,…,𝑚) in this paper are assumed to be Lipschitz continuous only, and not necessarily to be continuous differentiable, so the problem (𝑃) is a nonsmooth global optimization problem. Our filled function approach is proposed to tackle this kind of nonsmooth global optimization problem. Whereas these filled function methods proposed in the literature are mainly used for solving smooth global optimization problem.(2)Although the filled function presented in [2] can be extended to solve nonsmooth global optimization problem, it has some disadvantages, such as it has two dependent parameters which are difficult to determine and hence make its algorithmic realization be fairly complicated. However, our proposed filled function contains only one parameter, which is easier to be appropriately chosen, and thus overcomes the disadvantages mentioned above in a certain extent.(3)If each 𝑓_𝑖 is continuous differentiable and 𝑓(𝑥)=Σ^𝑚_𝑘=1𝑓²_𝑘(𝑥), then problem (𝑃) is a smooth optimization problem. In this case, our proposed filled function method can also be used to solve this smooth problem, just as the above-mentioned filled function methods can do. Therefore, our filled function method is a natural extension of these filled function methods in the literature, and it has more scope of applications in the field of global optimization.

The paper is organized as follows: in Section 2, we make several assumptions on the problem. In Section 3, we present a novel filled function and investigate its properties. In Section 4, we describe the filled function algorithm. In Section 5, we make a numerical test. And, at last, in Section 6, we give our conclusion.

2. Basic Knowledge

In this section, we give a summary of main results on nonsmooth analysis, and define filled function for the problem (𝑃). For more details, we refer the reader to [8].

Definition 2.1. Let 𝑓 be Lipschitz with constant 𝐿 at the point 𝑥, the generalized gradient of 𝑓 at 𝑥 is defined as 𝜕𝑓(𝑥)=𝜉∈𝑅^𝑛∶𝜉,𝑑≤𝑓⁰(𝑥;𝑑),∀𝑑∈𝑅^𝑛, (2.1) where 𝑓⁰(𝑥;𝑑)=limsup_{𝑦→𝑥,𝑡↓0}(𝑓(𝑦+𝑡𝑑)−𝑓(𝑦))/𝑡 is the generalized directional derivative of 𝑓(𝑥) in the direction 𝑑 at 𝑥.

Lemma 2.2. Let 𝑓 be Lipschitz with constant 𝐿 at the point 𝑥, then(a)𝑓⁰(𝑥;𝑑) is finite, sublinear and satisfies |𝑓⁰(𝑥;𝑑)|≤𝐿‖𝑑‖,(b)as a function of (𝑥,𝑑),𝑓⁰(𝑥;𝑑) is super-semicontinuous; as a function of 𝑑, it is Lipschitz with constant 𝐿,(c)𝜕Σ𝑠_𝑖𝑓_𝑖(𝑥)⊆Σ𝑠_𝑖𝜕𝑓_𝑖(𝑥), ∀𝑠_𝑖∈𝑅,(d)𝜕𝑓(𝑥)is a nonempty compact convex set, and to any 𝜉∈𝜕𝑓(𝑥), one has ‖𝜉‖≤𝐿,(e)∀𝑑∈𝑋, 𝑓⁰(𝑥;𝑑)=max{⟨𝜉,𝑑⟩∶𝜉∈𝜕𝑓(𝑥)}.

Consider the problem (𝑃). To extend the filled function for smooth global optimization to nonsmooth case, throughout this paper, we make the following assumptions.

Assumption 1. The function 𝑓_𝑖(𝑥)(𝑖=1,2,…,𝑚) is Lipschitz continuous on 𝑋 with a rank 𝐿_𝑖>0. Then 𝑓(𝑥) is Lipschitz continuous with a rank 𝐿=∑^𝑚_𝑖=1𝐿_𝑖.

Assumption 2. The value of 𝑓(𝑥) for 𝑥 on the boundary of 𝑋 is greater than that of 𝑓(𝑥) for any 𝑥 within 𝑋.

Note that Assumption 2 implies that all optimizers of (𝑃) lie in the interior of 𝑋.

Assumption 3. The problem (SNE) has at least one root.

Definition 2.3. A function 𝑃(𝑥,𝑥^∗) is called a filled function of 𝑓(𝑥) at a local minimizer 𝑥^∗, if it has the following properties: (1)𝑓(𝑥) is a strictly maximizer of 𝑃(𝑥,𝑥^∗),(2)for any 𝑥∈𝑆₁={𝑥∈𝑋⧵𝑥^∗∶𝑓(𝑥)≥𝑓(𝑥^∗)}, one has 0∉𝜕𝑃(𝑥,𝑥^∗),(3)if 𝑥^∗ is not a global minimizer of (𝑃), then 𝑃(𝑥,𝑥^∗) has at least one minimizer in the region 𝑆₂={𝑥∈𝑋∶𝑓(𝑥)<𝑓(𝑥^∗)}.

3. A New Filled Function and Its Properties

Let 𝑥^∗ be a local minimizer of (𝑃), 𝐿 the Lipschitz rank of 𝑓(𝑥), and 𝐷=max_{𝑥1,𝑥2∈𝑋}‖𝑥₁−𝑥₂‖. Define 𝐹𝑥,𝑥^∗=(𝑥)−1,𝑟arctan𝑓5arctan𝑓𝑥^∗𝑟exp‖𝑥−𝑥^∗‖+1, (3.1) where 𝑟>0 is a parameter.

Next, we will prove that 𝐹(𝑥,𝑥^∗,𝑟) is a filled function under mild conditions.

Lemma 3.1. Let 𝑔(𝑡)=(𝐿𝑡+(4/5)arctan𝑓(𝑥^∗))exp(𝑟/(𝑡+1)), where 0≤𝑡≤𝐷. If 𝑟 is an appropriately large such that 𝑟>(5𝐿(𝐷+1)²/4arctan𝑓(𝑥^∗)), then 𝑔(𝑡) is strictly monotone decreasing on [0,𝐷].

Proof. Since 𝑟>(5𝐿(𝐷+1)²/4arctan𝑓(𝑥^∗)), then we have 𝑔^′(𝑡)𝑟=exp4𝑡+1𝐿−𝐿𝑡+5arctan𝑓𝑥^∗𝑟(𝑡+1)²𝑟≤exp4𝑡+1𝐿−5arctan𝑓𝑥^∗𝑟(𝑡+1)²𝑟≤exp4𝑡+1𝐿−5arctan𝑓𝑥^∗𝑟(𝐷+1)²<0, (3.2) which implies that 𝑔^′(𝑡)<0, and 𝑔(𝑡) is strictly monotone decreasing on [0,𝐷].

Theorem 3.2. Let 𝑥^∗ be a local minimizer of 𝑓(𝑥) with 𝑓(𝑥^∗)>0. Suppose that Assumption 2 holds and 𝑟>0 is appropriately large such that 𝑟>(5𝐿(𝐷+1)²/4arctan𝑓(𝑥^∗)), then 𝑥^∗ is a strict local maximizer of 𝐹(𝑥,𝑥^∗,𝑟).

Proof. Since 𝑥^∗ is a local minimizer of 𝑓(𝑥) over 𝑋, there exists a neighborhood 𝑂(𝑥^∗,𝛿) of 𝑥^∗ with 𝛿>0 such that 𝑓(𝑥)≥𝑓(𝑥^∗) for all 𝑥∈𝑂(𝑥^∗,𝛿)⋂𝑋.
For any 𝑥∈𝑂(𝑥^∗,𝛿)⋂𝑋 with 𝑥≠𝑥^∗, if 𝑟>0 is appropriately large such that 𝑟>(5𝐿(𝐷+1)²/4arctan𝑓(𝑥^∗)), then, by Lemma 3.1 and the fact that the following inequality holds for 𝑏≥𝑎, arctan𝑏−arctan𝑎≤𝑏−𝑎, (3.3) we have that 𝐹𝑥,𝑥^∗𝑟,𝑟=exp‖𝑥−𝑥^∗‖(𝑥)−1+1arctan𝑓5arctan𝑓𝑥^∗𝑟≤exp‖𝑥−𝑥^∗‖𝑓(𝑥)𝑥+1−𝑓^∗+45arctan𝑓𝑥^∗𝑟≤exp‖𝑥−𝑥^∗‖𝐿+1𝑥−𝑥^∗+45arctan𝑓𝑥^∗(𝑟)4<exp5arctan𝑓𝑥^∗=𝐹𝑥,𝑥^∗,𝑟. (3.4) Therefore, 𝑥^∗ is a strict local maximizer of 𝐹(𝑥,𝑥^∗,𝑟).

Theorem 3.3. Let 𝑥^∗ be a local minimizer of 𝑓(𝑥) with 𝑓(𝑥^∗)>0. Suppose that 𝑟>0 is suitably large such that 𝑟>(5𝐿(𝐷+1)²/4arctan𝑓(𝑥^∗)), then 0∉𝜕𝐹(𝑥,𝑥^∗,𝑟), for all 𝑥∈𝑆₁.

Proof. For any 𝑥∈𝑆₁, that is, 𝑓(𝑥)≥𝑓(𝑥^∗) and 𝑥≠𝑥^∗, we have 𝜕𝐹𝑥,𝑥^∗𝑟,𝑟⊂exp‖𝑥−𝑥^∗‖1+11+𝑓²(𝑥)(𝑥)−(𝑥)−()𝑥𝜕𝑓arctan𝑓1/5arctan𝑓^∗(‖𝑥−𝑥^∗‖+1)²𝑟𝑥−𝑥^∗‖𝑥−𝑥^∗‖,𝜕𝐹𝑥,𝑥^∗,𝑟,𝑥−𝑥^∗‖𝑥−𝑥^∗‖𝑟⊂exp‖𝑥−𝑥^∗‖1+11+𝑓²(𝑥)𝜕𝑓(𝑥),𝑥−𝑥^∗‖𝑥−𝑥^∗‖(𝑥)−()𝑥−𝑟arctan𝑓1/5arctan𝑓^∗(‖𝑥−𝑥^∗‖+1)²𝑟≤exp‖𝑥−𝑥^∗‖()𝑥+1𝐿−𝑟4/5arctan𝑓^∗(𝐷+1)²<0, (3.5) since ‖𝜕𝑓(𝑥)‖≤𝐿, and 𝑟>(5𝐿(𝐷+1)²/4arctan𝑓(𝑥^∗)).
It follows that 0∉𝜕𝐹(𝑥,𝑥^∗,𝑟) for all 𝑥∈𝑆₁.

Theorem 3.4. Assume that Assumptions 2 and 3 hold, 𝑥^∗ is a local optimizer with 𝑓(𝑥^∗)>0 and is not a global optimizer of 𝑓(𝑥) over 𝑋. Then there exists a point 𝑥₀∈𝑆₂ such that 𝑥₀ is a minimizer of 𝐹(𝑥,𝑥^∗,𝑟) over 𝑋.

Proof. Since Assumption 3 holds and 𝑥^∗ is not a global minimizer of 𝑓(𝑥), there exists one point 𝑥^∗₁ with 𝑓(𝑥^∗₁)=0 such that 𝑓(𝑥^∗)>𝑓(𝑥^∗₁). Denote the boundary of 𝑋 by 𝜕𝑋. Then, by Assumption 2, for any 𝑥∈𝜕𝑋, it holds 𝐹(𝑥,𝑥^∗,𝑟)>𝐹(𝑥^∗₁,𝑥^∗,𝑟). Suppose that 𝐹(𝑥,𝑥^∗,𝑟) attains its minimum at the point 𝑥₀∈𝑋, then we have min_𝑥∈𝑋𝐹𝑥,𝑥^∗,𝑟=min_{𝑥∈𝑋⧵𝜕𝑋}𝐹𝑥,𝑥^∗,𝑟=𝐹𝑥₀,𝑥^∗,𝑟≤𝐹𝑥^∗₁,𝑥^∗,𝑟<0. (3.6) Since 𝑋⧵𝜕𝑋 is an open set, we have 𝑥₀∈𝑆₂. This completes the proof.

4. Solution Algorithm

Based on the discussion of the theoretical properties of the above-proposed filled function, we describe a filled function algorithm for solving (SNE) as follows.

Filled function algorithm:

Initialization Step
Let 𝑟_𝑈=10⁵ be a termination scalar. Choose an initial point 𝑥₁. Let 𝑒₁,𝑒₂,…,𝑒_2𝑛 be the coordinate directions. Set 𝑘=1, and go to the main step.

Main Step
(1)Start from 𝑥₁, solve the problem (𝑃) by using any nonsmooth local minimization procedure. Denote 𝑥^∗₁ the obtained optimizer, and go to (2).(2)Let 𝑟=1, construct the filled function 𝐹(𝑥,𝑥^∗,𝑟) and go to (3).(3)If 𝑘≤2𝑛, then set 𝑥=𝑥^∗₁+0.1𝑒_𝑘, and use 𝑥 as an initial point for a nonsmooth local minimization method to find a local minimizer 𝑥_𝑘 of the following problem: min_𝑥∈𝑋𝐹(𝑥,𝑥^∗,𝑟). If 𝑥 attains the boundary of 𝑋 during minimization, then set 𝑘=𝑘+1, and repeat (3); otherwise, go to (4).(4)If all the following conditions hold, then set 𝑥=𝑥_𝑘,𝑘=1, use 𝑥 as an initial point to get another local solution 𝑥^∗₂ of problem (𝑃), and go to (5); otherwise go to (6),(a)𝑓(𝑥_𝑘)<𝑓(𝑥^∗₁); (b)𝐹(𝑥_𝑘,𝑥^∗₁,𝑟)>𝐹(𝑥_𝑘−1,𝑥^∗₁,𝑟); (c)(𝑥_𝑘−𝑥^∗₁)^𝑇𝜉≥0,∀𝜉∈𝜕𝐹(𝑥_𝑘,𝑥^∗₁,𝑟); (d)‖𝜉‖<10⁻⁴,𝜉∈𝜕𝐹(𝑥_𝑘,𝑥^∗₁,𝑟). (5)If 𝑓(𝑥)<𝑓(𝑥^∗₁), then set 𝑥^∗₁=𝑥^∗₂, and go to (2); otherwise, go to (6).(6)Increase 𝑟 by setting 𝑟=10𝑟. If 𝑟≤𝑟_𝑈, then set 𝑘=1, and go to (2); otherwise, the algorithm is incapable of finding a better local minimizer, the algorithm stops, and 𝑥^∗₁ is taken as a global minimizer.

Remarks 4. The above filled function algorithm consists of two phases: local minimization and filling. The local minimization phase is intended to minimize problem (𝑃) by applying non-smooth local optimization algorithms, such as hybrid Hooke and Jeeves-Direct method for nonsmooth Optimization [9], mesh adaptive direct search algorithms for constrained optimization [10], bundle methods, and Powell's method. The filling phase aims to search for a point 𝑥_𝑘 with 𝑓(𝑥_𝑘)<𝑓(𝑥^∗₁) by minimizing the filled function 𝐹(𝑥,𝑥^∗,𝑟) over 𝑋. Once such kind of point is found in the process of minimization, the filling phase stops and the algorithm returns to the phase of local minimization. The two phases perform repeatedly until the certain conditions are satisfied.

5. Numerical Experiment

In this section, we perform the numerical experiments for four test problems by using the proposed filled function algorithm. All the numerical experiments are implemented in the FORTARN 95 environment. The hybrid Hooke and Jeeves-Direct method is used to search for the local minimizers of both the objective function and the filled function. Numerical results demonstrate that the proposed filled function approach is promising.

Problem 1. It holds that 4𝑥³₁+4𝑥₁𝑥₂+2𝑥²₂−42𝑥₁−14=0,4𝑥³₂+4𝑥₁𝑥₂+2𝑥²₁−26𝑥₂−22=0,s.t.−5≤𝑥₁,𝑥₂≤5. (5.1)

There are nine known roots as shown in [11]. Table 1 records the numerical results obtained by our algorithm for Problem 1.

Table 1 

Computational results for Problem 1.

Problem 2. It holds that 10⁴𝑥₁𝑥₂𝑒−1=0,_−𝑥1+𝑒^−𝑥2−1.001=0,s.t.5.49×10⁻⁶≤𝑥₁≤4.553,2.196×10⁻³≤𝑥₂≤18.21. (5.2)

The algorithm successfully finds a root of the Problem 2: (1.4564×10⁻⁵,6.8753). Table 2 records its numerical results.

Table 2 

Computational results for Problem 2.

Problem 3. It holds that 1−2𝑥₂+0.2sin4𝜋𝑥₂−𝑥₁𝑥=0,₂−0.5sin2𝜋𝑥₁=0,s.t.−10≤𝑥₁,𝑥₂≤10. (5.3)

The solution is (1.8784,−0.3458). Table 3 records the numerical results of Problem 3.

Table 3 

Computational results for Problem 3.

Problem 4. It holds that 2𝑥₁+𝑥₂+𝑥₃+𝑥₄+𝑥₅𝑥−6=0,₁+2𝑥₂+𝑥₃+𝑥₄+𝑥₅𝑥−6=0,₁+𝑥₂+2𝑥₃+𝑥₄+𝑥₅𝑥−6=0,₁+𝑥₂+𝑥₃+2𝑥₄+𝑥₅𝑥−6=0,₁𝑥₂𝑥₃𝑥₄𝑥₅𝑥−1=0,s.t._𝑖≤2,𝑖=1,2,…,5. (5.4)

The known solutions of Problem 4 in [11] are (1,1,1,1,1) and (0.916,0.916,0.916,0.916,0.916). Table 4 records the numerical results of Problem 4.

Table 4 

Computational results for Problem 4.

The symbols used in the tables are given below:𝑘:the iteration number in finding the 𝑘th local minimizer,𝑥_𝑘:the 𝑘th initial point to find the 𝑘th local minimizer,𝑥^∗_𝑘:the 𝑘th local minimizer,𝑓(𝑥^∗_𝑘):the function value of 𝑓(𝑥) at the 𝑘th local minimizer,𝐹(𝑥^∗_𝑘):the function value of 𝐹(𝑥) at the 𝑘th local minimizer.

6. Conclusions

In this paper, we developed a global minimization method based on filled function to identify the roots of the system of nonlinear equations. By converting a system nonlinear equations into an equivalent nonsmooth global minimization problem, we manage to search for a root or an appropriate root of the system of nonlinear equations by solving the formulated global minimization problem. A new filled function algorithm is proposed to solve the formulated global minimization problem. Numerical experiments for several test problems verify that our proposed algorithm is promising.

Acknowledgments

The authors are very grateful to the anonymous referees and the editor for their extremely valuable comments and suggestions, which have contributed to significant improvements of this paper. This paper was partially supported by the NNSF of China under Grant nos. 10971053 and 11001248, and by the SEDF under Grant no. 12YZ178.