Mathematical Problems in Engineering

Mathematical Problems in Engineering / 2012 / Article

Research Article | Open Access

Volume 2012 |Article ID 971468 |

Zhong Jin, Yuqing Wang, "A Nonmonotone Line Search Filter Algorithm for the System of Nonlinear Equations", Mathematical Problems in Engineering, vol. 2012, Article ID 971468, 15 pages, 2012.

A Nonmonotone Line Search Filter Algorithm for the System of Nonlinear Equations

Academic Editor: Wei-Chiang Hong
Received09 Jun 2012
Accepted23 Jun 2012
Published04 Sep 2012


We present a new iterative method based on the line search filter method with the nonmonotone strategy to solve the system of nonlinear equations. The equations are divided into two groups; some equations are treated as constraints and the others act as the objective function, and the two groups are just updated at the iterations where it is needed indeed. We employ the nonmonotone idea to the sufficient reduction conditions and filter technique which leads to a flexibility and acceptance behavior comparable to monotone methods. The new algorithm is shown to be globally convergent and numerical experiments demonstrate its effectiveness.

1. Introduction

We consider the following system of nonlinear equations: 𝑐𝑖(π‘₯)=0,𝑖=1,2,…,π‘š,(1.1) where each π‘π‘–βˆΆπ‘…π‘›β†’π‘…(𝑖=1,2,…,π‘š) is a twice continuously differentiable function. It is one of the most basic problems in mathematics and has lots of applications in many scientific fields such as physics, chemistry, and economics.

In the context of solving nonlinear equations, a well-known method is the Newton method, which is known to exhibit local and second order convergence near a regular solution, but its global behavior is unpredictable. To improve the global properties, some important algorithms [1] for nonlinear equations proceed by minimizing a least square problem: minβ„Ž(π‘₯)=𝑐(π‘₯)𝑇𝑐(π‘₯),(1.2) which can be also handled by the Newton method, while Powell [2] gives a counterexample to show a dissatisfactory fact that the iterates generated by the above least square problem may converge to a nonstationary point of β„Ž(π‘₯).

However, as we all know, there are several difficulties in utilizing the penalty functions as a merit function to test the acceptability of the iterates. Hence, the filter, a new concept first introduced by Fletcher and Leyffer [3] for constrained nonlinear optimization problems in a sequential quadratic programming (SQP) trust-region algorithm, replaces the merit fuctions avoiding the penalty parameter estimation and the difficulties related to the nondifferentiability. Furthermore, Fletcher et al. [4, 5] give the global convergence of the trust-region filter-SQP method, then Ulbrich [6] gets its superlinear local convergence. Consequently, filter method has been actually applied in many optimization techniques, for instance the pattern search method [7], the SLP method [8], the interior method [9], the bundle approaches [10, 11], and so on. Also combined with the trust-region search technique, Gould et al. extended the filter method to the system of nonlinear equations and nonlinear least squares in [12], and to the unconstrained optimization problem with multidimensional filter technique in [13]. In addition, WΓ€chter and Biegler [14, 15] presented line search filter methods for nonlinear equality-constrained programming and the global and local convergence were given.

In fact, filter method exhibits a certain degree of nonmonotonicity. The idea of nonmonotone technique can be traced back to Grippo et al. [16] in 1986, combined with the line search strategy. Due to its excellent numerical exhibition, many nonmonotone techniques have been developed in recent years, for example [17, 18]. Especially in [17], a nonmonotone line search multidimensional filter-SQP method for general nonlinear programming is presented based on the WΓ€chter and Biegler methods [14, 15].

Recently, some other ways were given to attack the problem (1.1) (see [19–23]). There are two common features in these papers; one is the filter approach is utilized, and the other is that the system of nonlinear equations is transformed into a constrained nonlinear programming problem and the equations are divided into two groups; some equations are treated as constraints and the others act as the objective function. And two groups of equations are updated at every iteration in those methods. For instance combined with the filter line search technique [14, 15], the system of nonlinear equations in [23] becomes the following optimization problem with equality constraints: minπ‘–βˆˆπ‘†1𝑐2𝑖(π‘₯)s.t.𝑐𝑗(π‘₯)=0,π‘—βˆˆπ‘†2.(1.3) The choice of two sets 𝑆1 and 𝑆2 are given as follows: for some positive constant 𝑛0>0, it is defined that 𝑐2𝑖1(π‘₯π‘˜)β‰₯𝑐2𝑖2(π‘₯π‘˜)β‰₯β‹―β‰₯𝑐2π‘–π‘š(π‘₯π‘˜), then 𝑆1={𝑖kβˆ£π‘˜β‰€π‘›0} and 𝑆2={π‘–π‘˜βˆ£π‘˜β‰₯𝑛0+1}.

In this paper we present an algorithm to solve the system of nonlinear equations, combining the nonmonotone technique and line search filter method. We also divide the equations into two groups; one contains the equations that are treated as equality constraints and the square of other equations is regarded as objective function. But different from those methods in [19–23], we just update the two groups at the iterations where it is needed indeed, which can make the scale of the calculation decrease in a certain degree. Another merit of our paper is to employ the nonmonotone idea to the sufficient reduction conditions and filter which leads to a flexibility and acceptance behavior comparable to monotone methods. Moreover, in our algorithm two groups of equations cannot be changed after an f-type iteration, thus in the case that |π’œ|<∞, the two groups are fixed after finite number of iterations. And the filter should not be updated after an f-type iteration, so naturally the global convergence is discussed, respectively, according to whether the number of updated filter is infinite or not. Furthermore, the global convergent property is induced under some reasonable conditions. In the end, numerical experiments show that the method in this paper is effective.

The paper is outlined as follows. In Section 2, we describe and analyze the nonmonotone line search filter method. In Section 3 we prove the global convergence of the proposed algorithm. Finally, some numerical tests are given in Section 4.

2. A Nonmonotone Line Search Filter Algorithm

Throughout this paper, we use the notations π‘šπ‘˜(π‘₯)=‖𝑐𝑆1(π‘₯)β€–22=βˆ‘π‘–βˆˆπ‘†1𝑐2𝑖(π‘₯) and πœƒπ‘˜(π‘₯)=‖𝑐𝑆2(π‘₯)β€–22=βˆ‘π‘–βˆˆπ‘†2𝑐2𝑖(π‘₯). In addition, we denote the set of indices of those iterations in which the filter has been augmented by π’œβŠ†β„•.

The linearization of the KKT condition of (1.3) at the π‘˜th iteration π‘₯π‘˜ is as follows: βŽ›βŽœβŽœβŽπ΅π‘˜π΄π‘˜π‘†2ξ‚€π΄π‘˜π‘†2𝑇0βŽžβŽŸβŽŸβŽ βŽ›βŽœβŽœβŽπ‘ π‘˜πœ†+π‘˜βŽžβŽŸβŽŸβŽ βŽ›βŽœβŽœβŽπ‘”=βˆ’π‘˜π‘π‘˜π‘†2⎞⎟⎟⎠,(2.1) where π΅π‘˜ is the Hessian or approximate Hessian matrix of 𝐿(π‘₯,πœ†)=π‘šπ‘˜(π‘₯)+πœ†π‘‡π‘π‘†2(π‘₯), π΄π‘˜π‘†2=βˆ‡π‘π‘†2(π‘₯π‘˜) and 𝑔(π‘₯π‘˜)=βˆ‡π‘šπ‘˜(π‘₯π‘˜). Then the iterate formation is π‘₯π‘˜(π›Όπ‘˜,𝑙)=π‘₯π‘˜+π›Όπ‘˜,π‘™π‘ π‘˜, where π‘ π‘˜ is the solution of (2.1) and π›Όπ‘˜,π‘™βˆˆ(0,1] is a step size chosen by line search.

Now we describe the nonmonotone Armijo rule. Let 𝑀 be a nonnegative integer. For each π‘˜, let π‘š(π‘˜) satisfy π‘š(0)=1,0β‰€π‘š(π‘˜)≀min{π‘š(π‘˜βˆ’1)+1,𝑀} for π‘˜β‰₯1. For fixed constants π›Ύπ‘š,π›Ύπœƒβˆˆ(0,1), we might consider a trial point to be acceptable, if it leads to sufficient progress toward either goal, that is, if πœƒπ‘˜ξ€·π‘₯π‘˜ξ€·π›Όπ‘˜,𝑙≀1βˆ’π›Ύπœƒξ€Έξƒ―πœƒmaxπ‘˜ξ€·π‘₯π‘˜ξ€Έ,π‘š(π‘˜)βˆ’1ξ“π‘Ÿ=0πœ†π‘˜π‘Ÿπœƒπ‘˜βˆ’π‘Ÿξ€·π‘₯π‘˜βˆ’π‘Ÿξ€Έξƒ°orπ‘šπ‘˜ξ€·π‘₯π‘˜ξ€·π›Όπ‘˜,π‘™ξƒ―π‘šξ€Έξ€Έβ‰€maxπ‘˜ξ€·π‘₯π‘˜ξ€Έ,π‘š(π‘˜)βˆ’1ξ“π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿξ€·π‘₯π‘˜βˆ’π‘Ÿξ€Έξƒ°βˆ’π›Ύπ‘šπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ,(2.2) where πœ†π‘˜π‘Ÿβˆˆ(0,1), βˆ‘π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†π‘˜π‘Ÿ=1.

For the convenience we set π‘š(π‘₯π‘˜)=max{π‘šπ‘˜(π‘₯π‘˜βˆ‘),π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿ(π‘₯π‘˜βˆ’π‘Ÿ)}, and πœƒ(π‘₯π‘˜)=max{πœƒπ‘˜(π‘₯π‘˜βˆ‘),π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†π‘˜π‘Ÿπœƒπ‘˜βˆ’π‘Ÿ(π‘₯π‘˜βˆ’π‘Ÿ)}. In order to avoid the case of convergence to a feasible but nonoptimal point, we consider the following switching condition: π‘”π‘‡π‘˜π‘ π‘˜<βˆ’πœ‰π‘ π‘‡π‘˜π΅π‘˜π‘ π‘˜,βˆ’π›Όπ‘˜,π‘™π‘”π‘‡π‘˜π‘ π‘˜>ξ€Ίπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έξ€»π‘ πœƒ,(2.3) with πœ‰βˆˆ(0,1],π‘ πœƒβˆˆ(0,1). If the switching condition holds, the trial point π‘₯π‘˜(π›Όπ‘˜,𝑙) has to satisfy the Armijo nonmonotone reduction condition, π‘šπ‘˜ξ€·π‘₯π‘˜ξ€·π›Όπ‘˜,π‘™β‰€ξ€Έξ€Έπ‘šξ€·π‘₯π‘˜ξ€Έ+𝜏3π›Όπ‘˜,π‘™π‘”π‘‡π‘˜π‘ π‘˜,(2.4) where 𝜏3∈(0,1/2) is a fixed constant.

To ensure the algorithm cannot cycle, it maintains a filter, a β€œtaboo region” β„±π‘˜βŠ†[0,∞]Γ—[0,∞] for each iteration π‘˜. The filter contains those combinations of constraint violation value πœƒ and the objective function value π‘š, that are prohibited for a successful trial point in iteration π‘˜. During the line search, a trial point π‘₯π‘˜(π›Όπ‘˜,𝑙) is rejected, if (πœƒ(π‘₯π‘˜(π›Όπ‘˜,𝑙)),π‘š(π‘₯π‘˜(π›Όπ‘˜,𝑙)))βˆˆβ„±π‘˜. We then say that the trial point is not acceptable to the current filter, which is also called π‘₯π‘˜(π›Όπ‘˜,𝑙)βˆˆβ„±π‘˜.

If a trial point π‘₯π‘˜(π›Όπ‘˜,𝑙)βˆ‰β„±π‘˜ satisfies the switching condition (2.3) and the reduction condition (2.4), then this trial point is called an f-type point, and accordingly this iteration is called an f-type iteration. An f-type point should be accepted as π‘₯π‘˜+1 with no updating of the filter, that is β„±π‘˜+1=β„±π‘˜.(2.5)

While if a trial point π‘₯π‘˜(π›Όπ‘˜,𝑙)βˆ‰β„±π‘˜ does not satisfy the switching condition (2.3), but this trial point satisfies (2.2), we call it an h-type point, or accordingly an h-type iteration. An h-type point should be accepted as π‘₯π‘˜+1 with updating of the filter, that is β„±π‘˜+1=β„±π‘˜ξšξ‚†(πœƒ,π‘š)βˆˆπ‘…2ξ€·βˆΆπœƒβ‰₯1βˆ’π›Ύπœƒξ€Έπœƒξ€·π‘₯π‘˜ξ€Έ,π‘šβ‰₯π‘šξ€·π‘₯π‘˜ξ€Έβˆ’π›Ύπ‘šπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έξ‚‡.(2.6)

In some cases it is not possible to find a trial step size that satisfies the above criteria. We approximate a minimum desired step size using linear models of the involved functions. For this, we define π›Όπ‘˜min=⎧βŽͺβŽͺβŽͺ⎨βŽͺβŽͺβŽͺβŽ©ξƒ―ξ€·min1βˆ’1βˆ’π›Ύπœƒξ€Έπœƒξ€·π‘₯π‘˜ξ€Έπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ,π‘šξ€·π‘₯π‘˜ξ€Έβˆ’π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έβˆ’π›Ύπ‘šπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έπ‘”π‘‡π‘˜π‘ π‘˜,ξ€Ίπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έξ€»π‘ πœƒβˆ’π‘”π‘‡π‘˜π‘ π‘˜ξƒ°,ifπ‘”π‘‡π‘˜π‘ π‘˜<βˆ’πœ‰π‘ π‘‡π‘˜π΅π‘˜π‘ π‘˜,ξ€·1βˆ’1βˆ’π›Ύπœƒξ€Έπœƒξ€·π‘₯π‘˜ξ€Έπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ,otherwise.(2.7) If the nonmonotone line search encounters a trial step size with π›Όπ‘˜,𝑙<π›Όπ‘˜min, the algorithm reverts to a feasibility restoration phase. Here, we try to find a new iterate which is acceptable to the current filter and for which (2.2) holds, by reducing the constraint violation with some iterative method.

The corresponding algorithm can be written as follows.

Algorithm 2.1. Step 1. Initialization: choose an initial guess π‘₯0, 𝜌1,𝜌2∈(0,1), 𝜌1<𝜌2, and πœ–>0. Compute 𝑔0, 𝑐𝑖(π‘₯0), 𝑆01, 𝑆02, and π΄π‘˜ for π‘–βˆˆπ‘†02. Set 𝑀>0, π‘š(0)=1, π‘˜=0, and β„±0=βˆ….
Step 2. If ‖𝑐(π‘₯π‘˜)β€–β‰€πœ– then stop.
Step 3. Compute (2.1) to obtain π‘ π‘˜. If there exists no solution to (2.1), go to Step 8. If β€–π‘ π‘˜β€–β‰€πœ– then stop.
Step 4. Use nonmonotone line search. Set 𝑙=0 and π›Όπ‘˜,𝑙=1.  Step 4.1. If π›Όπ‘˜,𝑙<π›Όπ‘˜min, where the π›Όπ‘˜min is obtained by (2.7), go to Step 8. Otherwise we get π‘₯π‘˜(π›Όπ‘˜,𝑙)=π‘₯π‘˜+π›Όπ‘˜,π‘™π‘ π‘˜. If π‘₯π‘˜(𝛼k,𝑙)βˆˆβ„±π‘˜, go to Step 4.3.  Step 4.2. Check sufficient decrease with respect to current iterate.  Step 4.2.1. If the switching condition (2.3) and the nonmonotone reduction condition (2.4) hold, set β„±π‘˜+1=β„±π‘˜ and go to Step 5. While only the switching condition (2.3) are satisfied, go to Step 4.3.  Step 4.2.2. The switching conditions (2.3) are not satisfied. If the nonmonotone filter condition (2.2) holds, set π‘₯π‘˜+1=π‘₯π‘˜+π›Όπ‘˜,π‘™π‘ π‘˜, augment the filter using (2.6) and go to Step 6. Otherwise, go to Step 4.3. Step 4.3. Choose π›Όπ‘˜,𝑙+1∈[𝜌1π›Όπ‘˜,𝑙,𝜌2π›Όπ‘˜,𝑙]. Let 𝑙=𝑙+1 and go to Step 4.1.
Step 5. Set π‘₯π‘˜+1=π‘₯π‘˜+π›Όπ‘˜,π‘™π‘ π‘˜, 𝑆1π‘˜+1=π‘†π‘˜1 and 𝑆2π‘˜+1=π‘†π‘˜2. Go to Step 7.
Step 6. Compute 𝑆1π‘˜+1 and 𝑆2π‘˜+1 by (1.3). If (πœƒπ‘˜+1(π‘₯π‘˜+1),π‘šπ‘˜+1(π‘₯π‘˜+1))βˆˆβ„±π‘˜+1, set 𝑆1π‘˜+1=π‘†π‘˜1 and 𝑆2π‘˜+1=π‘†π‘˜2.
Step 7. Compute π‘”π‘˜+1, π΅π‘˜+1, π΄π‘˜+1 and π‘š(π‘˜+1)=min{π‘š(π‘˜)+1,𝑀}. Let π‘˜=π‘˜+1 and go to Step 2.
Step 8 (restoration stage). Find π‘₯π‘Ÿπ‘˜=π‘₯π‘˜+π›Όπ‘Ÿπ‘˜π‘ π‘Ÿπ‘˜ such that π‘₯π‘Ÿπ‘˜ is acceptable to π‘₯π‘˜ and (πœƒπ‘˜(π‘₯π‘Ÿπ‘˜),π‘šπ‘˜(π‘₯π‘Ÿπ‘˜))βˆ‰β„±π‘˜. Set π‘₯π‘˜+1=π‘₯π‘Ÿπ‘˜ and augment the filter by (2.6). Let π‘˜=π‘˜+1, π‘š(π‘˜)=1 and go to Step 2.

In a restoration algorithm, the infeasibility is reduced and it is, therefore, desired to decrease the value of πœƒπ‘˜(π‘₯). The direct way is to utilize the Newton method or the similar ways to attack πœƒπ‘˜(π‘₯+𝑠)=0. We now give the restoration algorithm.

Restoration Algorithm
 Step R1. Let π‘₯0π‘˜=π‘₯π‘˜, 𝐻0=𝐸𝑛, Ξ”0π‘˜=Ξ”0, π‘”πœƒ=βˆ‡πœƒπ‘˜(π‘₯), 𝑗=0, πœ‚1=0.25, πœ‚2=0.75.  Step R2. If π‘₯π‘—π‘˜ is acceptable to π‘₯π‘˜ and (πœƒπ‘˜(π‘₯π‘Ÿπ‘˜),π‘šπ‘˜(π‘₯π‘Ÿπ‘˜))βˆ‰β„±π‘˜, then let π‘₯π‘Ÿπ‘˜=π‘₯π‘—π‘˜ and stop.  Step R3. Compute minπ‘”π‘‡πœƒ1𝑠+2𝑠𝑇𝐻𝑗𝑠s.t.β€–π‘ β€–β‰€Ξ”π‘—π‘˜(2.8) to get π‘ π‘—π‘˜. Let π‘Ÿπ‘—π‘˜=(πœƒπ‘˜(π‘₯π‘—π‘˜)βˆ’πœƒπ‘˜(π‘₯π‘—π‘˜+π‘ π‘—π‘˜))/(βˆ’π‘”π‘‡πœƒπ‘ π‘—π‘˜βˆ’(1/2)π‘ π‘—π‘˜π‘‡π»π‘—π‘ π‘—π‘˜).  Step R4. If π‘Ÿπ‘—π‘˜β‰€πœ‚1, set Ξ”π‘˜π‘—+1=(1/2)Ξ”π‘—π‘˜; If π‘Ÿπ‘—π‘˜β‰₯πœ‚2, set Ξ”π‘˜π‘—+1=2Ξ”π‘—π‘˜; otherwise, Ξ”π‘˜π‘—+1=Ξ”π‘—π‘˜. Let π‘₯π‘˜π‘—+1=π‘₯π‘—π‘˜+π‘ π‘—π‘˜, 𝐻𝑗 be updated to 𝐻𝑗+1, 𝑗=𝑗+1 and go to Step R2.

The above restoration algorithm is an SQP method for πœƒπ‘˜(π‘₯+𝑠)=0. Of course, there are other restoration algorithms, such as the Newton method, interior point restoration algorithm, SLP restoration algorithm, and so on.

3. Global Convergence of Algorithm

In this section, we present a proof of global convergence of Algorithm 2.1. We first state the following assumptions in technical terms.

Assumptions. (A1) All points π‘₯π‘˜ that are sampled by algorithm lie in a nonempty closed and bounded set 𝑋.
(A2) The functions 𝑐𝑖(π‘₯), 𝑗=1,2,…,π‘š are all twice continuously differentiable on an open set containing 𝑋.
(A3) There exist two constants 𝑏β‰₯π‘Ž>0 such that the matrices sequence {π΅π‘˜} satisfies π‘Žβ€–π‘ β€–2β‰€π‘ π‘‡π΅π‘˜π‘ β‰€π‘β€–π‘ β€–2 for all π‘˜ and π‘ βˆˆπ‘…π‘›.
(A4) (π΄π‘˜π‘ 2)𝑇 has full column rank and β€–π‘ π‘˜β€–β‰€π›Ύπ‘  for all π‘˜ with a positive constant 𝛾𝑠.

In the remainder of this section, we will not consider the case where Algorithm 2.1 terminates successfully in Step 2, since in this situation the global convergence is trivial.

Lemma 3.2. Under Assumption A1, there exists the solution to (2.1) with exact (or inexact) line search which satisfies the following descent conditions: ||πœƒπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’(1βˆ’2𝛼)πœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ||β‰€πœ1𝛼2β€–β€–π‘ π‘˜β€–β€–2,||π‘š(3.1)π‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’π‘šπ‘˜ξ€·π‘₯kξ€Έβˆ’π›Όπ‘”π‘‡π‘˜π‘ π‘˜||β‰€πœ2𝛼2β€–β€–π‘ π‘˜β€–β€–2,(3.2) where π›Όβˆˆ(0,1), 𝜏1 and 𝜏2 are all positive constants independent of π‘˜.

Proof. By virtue of the Taylor expansion of 𝑐2𝑖(π‘₯π‘˜+π›Όπ‘ π‘˜) with π‘–βˆˆπ‘†2, we obtain |||𝑐2𝑖π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’π‘2𝑖π‘₯π‘˜ξ€Έβˆ’2𝛼𝑐𝑖π‘₯π‘˜ξ€Έβˆ‡π‘π‘–ξ€·π‘₯π‘˜ξ€Έπ‘‡π‘ π‘˜|||=|||𝑐2𝑖π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’π‘2𝑖π‘₯π‘˜ξ€Έβˆ’2𝑐𝑖π‘₯π‘˜ξ€Έβˆ‡π‘π‘–ξ€·π‘₯π‘˜ξ€Έπ‘‡ξ€·π›Όπ‘ π‘˜ξ€Έ|||=|||12ξ€·π›Όπ‘ π‘˜ξ€Έπ‘‡ξ‚ƒ2𝑐𝑖π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έβˆ‡π‘2𝑖π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έ+2βˆ‡π‘π‘–ξ€·π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έβˆ‡π‘π‘–ξ€·π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έπ‘‡ξ‚„ξ€·π›Όπ‘ π‘˜ξ€Έ|||=|||𝛼2π‘ π‘‡π‘˜ξ‚ƒπ‘π‘–ξ€·π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έβˆ‡c2𝑖π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έ+βˆ‡π‘π‘–ξ€·π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έβˆ‡π‘π‘–ξ€·π‘₯π‘˜+πœπ›Όπ‘ π‘˜ξ€Έπ‘‡ξ‚„π‘ π‘˜|||≀1π‘šπœ1𝛼2β€–β€–π‘ π‘˜β€–β€–2,(3.3) where the last inequality can be done by Assumption A1 and 𝜁∈[0,1]. Furthermore, from (2.1) we immediately obtain 𝑐𝑖(π‘₯π‘˜)+βˆ‡π‘π‘–(π‘₯π‘˜)π‘‡π‘ π‘˜=0, that is, βˆ’2𝛼𝑐2𝑖(π‘₯π‘˜)βˆ’2𝛼𝑐𝑖(π‘₯π‘˜)βˆ‡π‘π‘–(π‘₯π‘˜)π‘‡π‘ π‘˜=0. With |𝑆2|β‰€π‘š, thereby, ||πœƒπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’(1βˆ’2𝛼)πœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ||=|||||ξ“π‘–βˆˆπ‘†2𝑐2𝑖π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’(1βˆ’2𝛼)𝑐2𝑖π‘₯π‘˜|||||β‰€ξ“ξ€Έξ€Έπ‘–βˆˆπ‘†2||𝑐2𝑖π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’(1βˆ’2𝛼)𝑐2𝑖π‘₯π‘˜ξ€Έ||=ξ“π‘–βˆˆπ‘†2|||𝑐2𝑖π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’π‘2𝑖π‘₯π‘˜ξ€Έβˆ’2𝛼𝑐𝑖π‘₯π‘˜ξ€Έβˆ‡π‘π‘–ξ€·π‘₯π‘˜ξ€Έπ‘‡π‘ π‘˜|||1β‰€π‘šβ‹…π‘šπœ1𝛼2β€–β€–π‘ π‘˜β€–β€–2β‰€πœ1𝛼2β€–β€–π‘ π‘˜β€–β€–2,(3.4) then the first inequality consequently holds.
According to the Taylor expansion of βˆ‘π‘–βˆˆπ‘†1(𝑐2𝑖(π‘₯π‘˜+π›Όπ‘ π‘˜)) (i.e., π‘šπ‘˜(π‘₯π‘˜+π›Όπ‘ π‘˜)), we then have |||||ξ“π‘–βˆˆπ‘†1𝑐2𝑖π‘₯π‘˜+π›Όπ‘ π‘˜βˆ’ξ“ξ€Έξ€Έπ‘–βˆˆπ‘†1𝑐2𝑖π‘₯π‘˜ξ€Έξ€Έβˆ’π›Όπ‘”π‘‡π‘˜π‘ π‘˜|||||=|||||12𝛼2ξ€·π‘ π‘˜ξ€Έπ‘‡βˆ‡2ξ“π‘–βˆˆπ‘†1𝑐2𝑖π‘₯π‘˜+πœšπ›Όπ‘ π‘˜π‘ ξ€Έξ€Έπ‘˜|||||β‰€πœ2𝛼2β€–β€–π‘ π‘˜β€–β€–2,(3.5) where the last inequality follows from Assumption A1 and 𝜚∈[0,1]. That is to say, ||π‘šπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έβˆ’π›Όπ‘”π‘‡π‘˜π‘ π‘˜||β‰€πœ2𝛼2β€–β€–π‘ π‘˜β€–β€–2,(3.6) which is just (3.2).

Lemma 3.3. Let {π‘₯π‘˜π‘–} be a subsequence of iterates for which (2.3) holds and has the same 𝑆1 and 𝑆2. Then there exists some ξπ›Όβˆˆ(0,1] such that π‘šπ‘˜π‘–ξ€·π‘₯π‘˜π‘–+ξπ›Όπ‘ π‘˜π‘–ξ€Έβ‰€π‘šπ‘˜π‘–ξ€·π‘₯π‘˜π‘–ξ€Έ+ξπ›Όπœ3π‘”π‘‡π‘˜π‘–π‘ π‘˜π‘–.(3.7)

Proof. Because {π‘₯π‘˜π‘–} have the same 𝑆1 and 𝑆2, it follows that π‘šπ‘˜π‘–(π‘₯) are fixed and by (2.3) π‘‘π‘˜π‘– is a decent direction. Hence there exists some ξπ›Όβˆˆ(0,1] satisfying (3.7).

Theorem 3.4. Suppose that {π‘₯π‘˜} is an infinite sequence generated by Algorithm 2.1 and |π’œ|<∞, one has limπ‘˜β†’βˆžβ€–β€–β€–π‘π‘˜π‘†π‘˜2β€–β€–β€–+β€–β€–π‘ π‘˜β€–β€–=0,(3.8) namely, every limit point is the πœ– solution to (1.1) or a local infeasible point. If the gradients of 𝑐𝑖(π‘₯π‘˜) are linear independent for all π‘˜ and 𝑖=1,2,…,π‘š, then the solution to SNE is obtained.

Proof. From |π’œ|<∞, we know the filter updates in a finite number, then there exists πΎβˆˆβ„•, for π‘˜>𝐾 the filter does not update. As h-type iteration and restoration algorithm all need the updating of the filter, so for π‘˜>𝐾 our algorithm only follows the f-type iterations. We then have that for all π‘˜>𝐾 both conditions (2.3) and (2.4) are satisfied for π‘₯π‘˜+1=π‘₯π‘˜+π›Όπ‘˜π‘ π‘˜ and π‘šπ‘˜(π‘₯)=π‘šπΎ(π‘₯).
Then by (2.4) we get π‘šπ‘˜(π‘₯π‘˜+1)≀max{π‘šπ‘˜(π‘₯π‘˜βˆ‘),π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿ(π‘₯π‘˜βˆ’π‘Ÿ)}+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜. We first show that for all π‘˜β‰₯𝐾+1, it holds π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έ<π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’2ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜βˆ’1π‘”π‘‡π‘˜βˆ’1π‘ π‘˜βˆ’1<π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’1ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ,(3.9) where π‘š(π‘₯𝐾)=max{π‘šπΎ(π‘₯πΎβˆ‘),π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†πΎπ‘Ÿπ‘šπΎβˆ’π‘Ÿ(π‘₯πΎβˆ’π‘Ÿ)}. We prove (3.9) by induction.
If π‘˜=𝐾+1, we have π‘šπΎ+1(π‘₯𝐾+1)<π‘š(π‘₯𝐾)+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜<π‘š(π‘₯𝐾)+πœ†πœ3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜. Suppose that the claim is true for 𝐾+1,𝐾+2,…,π‘˜, then we consider two cases.
Case 1. If max{π‘šπ‘˜(π‘₯π‘˜βˆ‘),π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿ(π‘₯π‘˜βˆ’π‘Ÿ)}=π‘šπ‘˜(π‘₯π‘˜), it is clear that π‘šπ‘˜+1ξ€·π‘₯π‘˜+1ξ€Έ<π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έ+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜<π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’1ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜β‰€π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ.(3.10)
Case 2. If max{π‘šπ‘˜(π‘₯π‘˜βˆ‘),π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿ(π‘₯π‘˜βˆ’π‘Ÿβˆ‘)}=π‘š(π‘˜)βˆ’1π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿ(π‘₯π‘˜βˆ’π‘Ÿ), let 𝑒=π‘š(π‘˜)βˆ’1. By the fact thatβ€‰β€‰βˆ‘π‘’π‘‘=0πœ†π‘˜π‘‘=1, πœ†β‰€πœ†π‘˜π‘‘<1, we have π‘šπ‘˜+1ξ€·π‘₯π‘˜+1ξ€Έ<𝑒𝑑=0πœ†π‘˜π‘‘π‘šπ‘˜βˆ’π‘‘ξ€·π‘₯π‘˜βˆ’π‘‘ξ€Έ+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜<𝑒𝑑=0πœ†π‘˜π‘‘ξƒ©π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’π‘‘βˆ’2ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜βˆ’π‘‘βˆ’1π‘”π‘‡π‘˜βˆ’π‘‘βˆ’1π‘ π‘˜βˆ’π‘‘βˆ’1ξƒͺ+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜=πœ†π‘˜0ξƒ©π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’π‘’βˆ’2ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+πœ†πœ3π‘˜βˆ’2ξ“π‘Ÿ=π‘˜βˆ’π‘’βˆ’1π›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜βˆ’1π‘”π‘‡π‘˜βˆ’1π‘ π‘˜βˆ’1ξƒͺ+πœ†π‘˜1ξƒ©π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’π‘’βˆ’2ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+πœ†πœ3π‘˜βˆ’3ξ“π‘Ÿ=π‘˜βˆ’π‘’βˆ’1π›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜βˆ’2π‘”π‘‡π‘˜βˆ’2π‘ π‘˜βˆ’2ξƒͺ+β‹―+πœ†π‘˜π‘’ξƒ©π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’π‘’βˆ’2ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜βˆ’π‘’βˆ’1π‘”π‘‡π‘˜βˆ’π‘’βˆ’1π‘ π‘˜βˆ’π‘’βˆ’1ξƒͺ+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜<π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’π‘’βˆ’2ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+πœ†πœ3π‘˜βˆ’1ξ“π‘Ÿ=π‘˜βˆ’π‘’βˆ’1π›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜=π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜βˆ’1ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ+𝜏3π›Όπ‘˜π‘”π‘‡π‘˜π‘ π‘˜<π‘šξ€·π‘₯𝐾+πœ†πœ3π‘˜ξ“π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ.(3.11) Moreover, since π‘šπ‘˜(π‘₯π‘˜) is bounded below as π‘˜β†’βˆž, we get βˆ‘π‘˜π‘Ÿ=πΎπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ<∞, that is, limπ‘˜β†’βˆžπ›Όπ‘Ÿπ‘”π‘‡π‘Ÿπ‘ π‘Ÿ=0. By Lemma 3.3, there exists a ξπ›Όβˆˆ(0,1] such that π›Όπ‘˜β‰₯𝛼. Then together with π‘”π‘‡π‘˜π‘ π‘˜<βˆ’πœ‰π‘ π‘‡π‘˜π΅π‘˜π‘ π‘˜ and Assumption A1, we have limπ‘˜β†’βˆžβ€–π‘ π‘˜β€–=0. From βˆ’π›Όπ‘˜,π‘™π‘”π‘‡π‘˜π‘ π‘˜>[πœƒπ‘˜(π‘₯π‘˜)]π‘ πœƒ it is easy to obtain that limπ‘˜β†’βˆžπœƒπ‘˜(π‘₯π‘˜)=0. This completes the proof.

Lemma 3.5. Under Assumptions A1 and A2, if π‘”π‘‡π‘˜π‘ π‘˜β‰€βˆ’πœ€0 for a positive constant πœ€0 independent of π‘˜(βˆˆπ‘Žπ‘ π‘’π‘π‘ π‘’π‘žπ‘’π‘’π‘›π‘π‘’) and for all π›Όβˆˆ(0,1] and 𝛼β‰₯𝛼minπ‘˜,𝑙 with (πœƒπ‘˜(π‘₯π‘˜),π‘šπ‘˜(π‘₯π‘˜))βˆ‰β„±π‘˜, then there exists 𝛾1, 𝛾2>0 so that (πœƒπ‘˜(π‘₯π‘˜+π›Όπ‘ π‘˜),π‘šπ‘˜(π‘₯π‘˜+π›Όπ‘ π‘˜))βˆ‰β„±π‘˜ for all π‘˜(βˆˆπ‘Žπ‘ π‘’π‘π‘ π‘’π‘žπ‘’π‘’π‘›π‘π‘’) and 𝛼≀min{𝛾1,𝛾2πœƒπ‘˜(π‘₯π‘˜)}.

Proof. Choose 𝛾1=πœ€0/𝜏2𝛾2𝑠, then 𝛼≀𝛾1 implies that βˆ’π›Όπœ€0+𝜏2𝛼2𝛾2𝑠≀0. So we note from (3.2) that π‘šπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβ‰€π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έ+π›Όπ‘”π‘‡π‘˜π‘ π‘˜+𝜏2𝛼2β€–β€–π‘ π‘˜β€–β€–2β‰€π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έβˆ’π›Όπœ€0+𝜏2𝛼2𝛾2π‘ β‰€π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έ.(3.12)
Let 𝛾2=2/𝜏1𝛾2𝑠, then 𝛼≀𝛾2πœƒπ‘˜(π‘₯π‘˜) implies that βˆ’2π›Όπœƒπ‘˜(π‘₯π‘˜)+𝜏1𝛼2𝛾2𝑠≀0. So from (3.1), we obtain πœƒπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβ‰€πœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έβˆ’2π›Όπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ+𝜏2𝛼2β€–β€–π‘ π‘˜β€–β€–2β‰€πœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έβˆ’2π›Όπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ+𝜏1𝛼2𝛾2π‘ β‰€πœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έ.(3.13)
We further point a fact according to the definition of filter. If (πœƒ,π‘š)βˆ‰β„±π‘˜ and πœƒβ‰€πœƒ, π‘šβ‰€π‘š, we obtain (πœƒ,π‘š)βˆ‰β„±π‘˜. Thus from (πœƒπ‘˜(π‘₯π‘˜),π‘šπ‘˜(π‘₯π‘˜))βˆ‰β„±π‘˜, π‘šπ‘˜(π‘₯π‘˜+π›Όπ‘ π‘˜)β‰€π‘šπ‘˜(π‘₯π‘˜), and πœƒπ‘˜(π‘₯π‘˜+π›Όπ‘ π‘˜)β‰€πœƒπ‘˜(π‘₯π‘˜), we have (πœƒπ‘˜(π‘₯π‘˜+π›Όπ‘ π‘˜),π‘šπ‘˜(π‘₯π‘˜+π›Όπ‘ π‘˜))βˆ‰β„±π‘˜.

Lemma 3.6. If π‘”π‘‡π‘˜π‘ π‘˜β‰€βˆ’πœ€0 for a positive constant πœ€0 independent of π‘˜(βˆˆπ‘Žπ‘ π‘’π‘π‘ π‘’π‘žπ‘’π‘’π‘›π‘π‘’), then there exists a constant 𝛼>0, for all π‘˜(βˆˆπ‘Žπ‘ π‘’π‘π‘ π‘’π‘žπ‘’π‘’π‘›π‘π‘’) and 𝛼≀𝛼 such that π‘šπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έξƒ―π‘šβˆ’maxπ‘˜ξ€·π‘₯π‘˜ξ€Έ,π‘š(π‘˜)βˆ’1ξ“π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿξ€·π‘₯π‘˜βˆ’π‘Ÿξ€Έξƒ°β‰€πœ3π›Όπ‘”π‘‡π‘˜π‘ π‘˜.(3.14)

Proof. Let 𝛼=(1βˆ’πœ3)πœ€0/𝜏2𝛾2𝑠. In view of (3.2), β€–π‘ π‘˜β€–β‰€π›Ύπ‘  and 𝛼≀𝛼, we know π‘šπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έξƒ―π‘šβˆ’maxπ‘˜ξ€·π‘₯π‘˜ξ€Έ,π‘š(π‘˜)βˆ’1ξ“π‘Ÿ=0πœ†π‘˜π‘Ÿπ‘šπ‘˜βˆ’π‘Ÿξ€·π‘₯π‘˜βˆ’π‘Ÿξ€Έξƒ°βˆ’π›Όπ‘”π‘‡π‘˜π‘ π‘˜β‰€π‘šπ‘˜ξ€·π‘₯π‘˜+π›Όπ‘ π‘˜ξ€Έβˆ’π‘šπ‘˜ξ€·π‘₯π‘˜ξ€Έβˆ’π›Όπ‘”π‘‡π‘˜π‘ π‘˜β‰€πœ2𝛼2β€–β€–π‘ π‘˜β€–β€–2β‰€πœ2𝛼𝛼𝛾2𝑠=ξ€·1βˆ’πœ3ξ€Έπ›Όπœ€0ξ€·β‰€βˆ’1βˆ’πœ3ξ€Έπ›Όπ‘”π‘‡π‘˜π‘ π‘˜,(3.15) which shows that the assertion of the lemma follows.

Theorem 3.7. Suppose that {π‘₯π‘˜} is an infinite sequence generated by Algorithm 2.1 and |π’œ|=∞. Then there exists at least one accumulation which is the πœ– solution to (1.1) or a local infeasible point. Namely, one has limπ‘˜β†’βˆžξ‚ƒβ€–β€–β€–π‘infπ‘˜π‘†π‘˜2β€–β€–β€–+β€–β€–π‘ π‘˜β€–β€–ξ‚„=0.(3.16) If the gradients of 𝑐𝑖(π‘₯π‘˜) are linear independent for all π‘˜ and 𝑖=1,2,…,π‘š, then the solution to (1.1) is obtained.

Proof. We prove that limπ‘˜β†’βˆž,π‘˜βˆˆπ’œπœƒπ‘˜(π‘₯π‘˜)=0 first.
Suppose by contradiction that there exits an infinite subsequence {π‘˜π‘–} of π’œ such that πœƒπ‘˜π‘–(π‘₯π‘˜π‘–)β‰₯πœ€ for some πœ€>0. At each iteration π‘˜π‘–, (πœƒπ‘˜π‘–(π‘₯π‘˜π‘–),π‘šπ‘˜π‘–(π‘₯π‘˜π‘–)) is added to the filter which means that no other (πœƒ,π‘š) can be added to the filter at a later stage within the area: ξ‚ƒπœƒξ€·π‘₯π‘˜π‘–ξ€Έβˆ’π›Ύπœƒπœ€,πœƒξ€·π‘₯π‘˜π‘–ξ€Έξ‚„Γ—ξ€Ίπ‘šξ€·π‘₯π‘˜π‘–ξ€Έβˆ’π›Ύπ‘šπœ€,π‘šξ€·π‘₯π‘˜π‘–ξ€Έξ€»,(3.17) and the area of the each of these squares is at least π›Ύπœƒπ›Ύπ‘šπœ€2.
By Assumption A1 we have βˆ‘π‘›π‘–=1𝑐2𝑖(π‘₯π‘˜)≀𝑀max. Since 0β‰€π‘šπ‘˜(π‘₯π‘˜)β‰€π‘šπ‘˜(π‘₯π‘˜)+πœƒπ‘˜(π‘₯π‘˜βˆ‘)=𝑛𝑖=1𝑐2𝑖(π‘₯π‘˜) and 0β‰€πœƒπ‘˜(π‘₯π‘˜)β‰€π‘šπ‘˜(π‘₯π‘˜)+πœƒπ‘˜(π‘₯π‘˜βˆ‘)=𝑛𝑖=1𝑐2𝑖(π‘₯π‘˜), then (πœƒ,π‘š) associated with the filter are restricted to ℬ=0,𝑀maxξ€»Γ—ξ€Ί0,𝑀maxξ€».(3.18)
Thereby ℬ is completely covered by at most a finite number of such areas in contraction to the infinite subsequence {π‘˜π‘–} satisfying πœƒπ‘˜π‘–(π‘₯π‘˜π‘–)β‰₯πœ€. Therefore, limπ‘˜β†’βˆž,π‘˜βˆˆπ’œπœƒπ‘˜(π‘₯π‘˜)=0.
By Assumption A1 and |π’œ|=∞, there exits an accumulation point π‘₯, that is, limπ‘–β†’βˆžπ‘₯π‘˜π‘–=π‘₯, π‘˜π‘–βˆˆπ’œ. It follows from limπ‘˜β†’βˆž,π‘˜βˆˆπ’œπœƒπ‘˜(π‘₯π‘˜)=0, that limπ‘–β†’βˆžπœƒπ‘˜π‘–ξ€·π‘₯π‘˜π‘–ξ€Έ=0,(3.19) which implies limπ‘–β†’βˆžβ€–π‘π‘˜π‘–π‘†π‘˜π‘–2β€–=0. If limπ‘–β†’βˆžβ€–π‘ π‘˜π‘–β€–=0, then (3.16) is true. Otherwise, there exists a subsequence {π‘₯π‘˜π‘–π‘—} of {π‘₯π‘˜π‘–} and a constant πœ€1>0 so that for all π‘˜π‘–π‘—, β€–β€–π‘ π‘˜π‘–π‘—β€–β€–β‰₯πœ€1.(3.20) The choice of {π‘₯π‘˜π‘–π‘—} implies π‘˜π‘–π‘—βˆˆπ’œforallπ‘˜π‘–π‘—.(3.21) According to β€–π‘ π‘˜π‘–π‘—β€–β‰₯πœ€1, Assumption A1 as well as πœ‰βˆˆ(0,1), we have π‘”π‘‡π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—+πœ‰π‘ π‘‡π‘˜π‘–π‘—π΅π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—=(πœ‰βˆ’1)π‘ π‘‡π‘˜π‘–π‘—π΅π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—βˆ’ξ‚€πœ†+π‘˜π‘–π‘—ξ‚π‘‡π‘π‘˜π‘–π‘—π‘†π‘˜π‘–π‘—2‖‖𝑠≀(πœ‰βˆ’1)π‘Žπ‘˜π‘–π‘—β€–β€–2+𝑐1β€–β€–β€–π‘π‘˜π‘–π‘—π‘†π‘˜π‘–π‘—2‖‖‖≀(πœ‰βˆ’1)π‘Žπœ€21+𝑐1β€–β€–β€–π‘π‘˜π‘–π‘—π‘†π‘˜π‘–π‘—2β€–β€–β€–.(3.22) Since πœ‰βˆ’1<0 and β€–π‘π‘˜π‘–π‘—π‘†π‘˜π‘–π‘—2β€–β†’0 as π‘—β†’βˆž, we obtain π‘”π‘‡π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—β‰€βˆ’πœ‰π‘ π‘‡π‘˜π‘–π‘—π΅π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—,(3.23) for sufficiently large 𝑗. Similarly, we have π›Όπ‘”π‘‡π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—+ξ€Ίπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έξ€»π‘ πœƒβ‰€βˆ’π›Όπ‘ π‘‡π‘˜π‘–π‘—π΅π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—+𝑐1β€–β€–β€–π‘π‘˜π‘–π‘—π‘˜π‘–π‘—2β€–β€–β€–+ξ€Ίπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έξ€»π‘ πœƒβ‰€βˆ’π›Όπ‘Žπœ€21+𝑐1β€–β€–β€–π‘π‘˜π‘–π‘—π‘˜π‘–π‘—2β€–β€–β€–+ξ€Ίπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έξ€»π‘ πœƒ,(3.24) and thus βˆ’π›Όπ‘”π‘‡π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—β‰₯ξ€Ίπœƒπ‘˜ξ€·π‘₯π‘˜ξ€Έξ€»π‘ πœƒ,(3.25) for sufficiently large 𝑗. This means the condition (2.3) is satisfied for sufficiently large 𝑗. Therefore, the reason for accepting π‘₯π‘˜+1 must been that π‘₯π‘˜+1 satisfies nonmonotone Armijo condition (2.4). In fact let πœ€0=πœ‰π‘Žπœ€21, then π‘”π‘‡π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—β‰€βˆ’πœ‰π‘ π‘‡π‘˜π‘–π‘—π΅π‘˜π‘–π‘—π‘ π‘˜π‘–π‘—β‰€βˆ’πœ‰π‘Žπœ€21=βˆ’πœ€0; by Lemma 3.6 we obtain nonmonotone Armijo condition (2.4) is satisfied. Consequently, the filter is not augmented in iteration π‘˜π‘–π‘— which is a contraction to (3.21). The whole proof is completed.

4. Numerical Experiments

In this section, we test our algorithm on some typical test problems. In the whole process, the program is coded in MATLAB and we assume the error tolerance πœ– in this paper is always 1.0π‘’βˆ’5. The selected parameter values are π›Ύπœƒ=0.1, π›Ύπ‘š=0.1, π‘ πœƒ=0.9, 𝜌1=0.25, 𝜌2=0.75, and 𝑀=3. In the following tables, the notations NIT, NOF, and NOG mean the number of iterates, number of functions, and number of gradients, respectively.

Example 4.1. Find a solution of the nonlinear equations system as follows: βŽ›βŽœβŽœβŽπ‘₯+3𝑦2(⎞⎟⎟⎠=βŽ›βŽœβŽœβŽ00⎞⎟⎟⎠π‘₯βˆ’1.0)𝑦.(4.1)
The only solution of Example 4.1 is (π‘₯βˆ—,π‘¦βˆ—)=(0,0). Define the line Ξ“={(1,𝑦)βˆΆπ‘¦βˆˆβ„}. If the starting point (π‘₯0,𝑦0)βˆˆΞ“, the Newton method [24] are confined to Ξ“. We choose two starting points which belong to Ξ“ in the experiments and then the (π‘₯βˆ—,π‘¦βˆ—) is obtained. Table 1 shows the results.

Starting point NIT NOF NOG

( 1 , 0 )257
( 1 , 2 )6 1312

Example 4.2. Consider the system of nonlinear equations: 𝑐1(π‘₯)=π‘₯31βˆ’π‘₯32+π‘₯33π‘βˆ’1,2(π‘₯)=π‘₯21+π‘₯22βˆ’π‘₯23π‘βˆ’1,3(π‘₯)=π‘₯1+π‘₯2+π‘₯3βˆ’3.(4.2)
The solution to Example 4.2 is π‘₯βˆ—=(1,1,1)𝑇. The numerical results of Example 4.2 are given in Table 2.

Starting point NIT NOF NOG

( 0 , 0 , 0 )102122
( 1 . 5 , 1 . 5 , 1 . 5 )7 1515

Example 4.3. Find a solution of the nonlinear equations system: βŽ›βŽœβŽœβŽπ‘₯10π‘₯(π‘₯+0.1)+2𝑦2⎞⎟⎟⎠=βŽ›βŽœβŽœβŽ00⎞⎟⎟⎠.(4.3)
The unique solution is (π‘₯βˆ—,π‘¦βˆ—)=(0,0). It has been proved in [2] that, under initial point (π‘₯0,𝑦0)=(3,1), the iterates converge to the point 𝑧=(1.8016,0.0000), which is not a stationary point. Utilizing our algorithm, a sequence of points converging to (π‘₯βˆ—,π‘¦βˆ—) is obtained. The detailed numerical results for Example 4.3 are listed in Table 3.

Starting point NIT NOF NOG NIT [22]

( 3 , 1 )610817
( 3 0 , 1 0 )7 14137
( 3 0 0 , 1 0 0 )10 171615

Example 4.4. Consider the following system of nonlinear equations: 𝑐1(π‘₯)=π‘₯21+π‘₯1π‘₯2+2π‘₯22βˆ’π‘₯1βˆ’π‘₯2π‘βˆ’2,2(π‘₯)=2π‘₯21+π‘₯1π‘₯2+3π‘₯22βˆ’π‘₯1βˆ’π‘₯2βˆ’4.(4.4)
There are three solutions of above example, (1,1)𝑇, (βˆ’1,1)𝑇, and (1,βˆ’1)𝑇. The numerical results of Example 4.4 are given in Table 4.

Starting point( 0 . 5 , 0 . 5 )( βˆ’ 0 . 5 , 0 . 5 ) ( 0 . 5 , βˆ’ 0 . 5 )

NOF8 79
NOG7 69
Solution( 1 , 1 )( βˆ’ 1 , 1 )( 1 , βˆ’ 1 )

NIT [22]569

Example 4.5. Consider the system of nonlinear equations: 𝑐𝑖(π‘₯)=βˆ’(𝑁+1)+2π‘₯𝑖+𝑁𝑗=1,𝑗≠𝑖π‘₯𝑗𝑐,𝑖=1,2,…,π‘βˆ’1,(4.5)𝑁(π‘₯)=βˆ’1+𝑁𝑗=1π‘₯𝑗,(4.6) with the initial point π‘₯𝑖(0)=0.5, 𝑖=1,2,…,𝑁. The solution to Example 4.5 is π‘₯βˆ—=(1,1,…,1)𝑇. The numerical results of Example 4.5 are given in Table 5.

𝑁 = 1 0 𝑁 = 2 0 𝑁 = 4 0 𝑁 = 6 0 𝑁 = 1 2 0

NOF14 22263977
NOG13 21233468

NIT [22]8172241Fail

Refer to these above problems, running the Algorithm 2.1 with different starting points yields the results in the corresponding tables, which, summarized, show that our proposed algorithm is practical and effective. From the computation efficiency, we should point out our algorithm is competitive with the method in [22]. The results in Table 5 in fact show that our method also succeeds well to solve the cases when more equations are active.

Constrained optimization approaches attacking the system of nonlinear equations are exceedingly interesting and are further developed by using the nonmonotone line search filter strategy in this paper. Moreover, the local property of the algorithm is a further topic of interest.


The research is supported by the National Natural Science Foundation of China (no. 11126060) and Science & Technology Program of Shanghai Maritime University (no. 20120060).


  1. J. Nocedal and S. J. Wright, Numerical Optimization, Springer Series in Operations Research, Springer, New York, NY, USA, 1999. View at: Publisher Site | Zentralblatt MATH
  2. M. J. D. Powell, β€œA hybrid method for nonlinear equations,” in Numerical Methods for Nonlinear Algebraic Equations, P. Rabinowitz, Ed., pp. 87–114, Gordon and Breach, London, UK, 1970. View at: Google Scholar | Zentralblatt MATH
  3. R. Fletcher and S. Leyffer, β€œNonlinear programming without a penalty function,” Mathematical Programming, vol. 91, no. 2, pp. 239–269, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  4. R. Fletcher, S. Leyffer, and P. L. Toint, β€œOn the global convergence of a filter-SQP algorithm,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 44–59, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  5. R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A. Wächter, β€œGlobal convergence of a trust-region SQP-filter algorithm for general nonlinear programming,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 635–659, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  6. S. Ulbrich, β€œOn the superlinear local convergence of a filter-SQP method,” Mathematical Programming, vol. 100, no. 1, pp. 217–245, 2004. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  7. C. Audet and J. E. Dennis Jr., β€œA pattern search filter method for nonlinear programming without derivatives,” SIAM Journal on Optimization, vol. 14, no. 4, pp. 980–1010, 2004. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  8. C. M. Chin and R. Fletcher, β€œOn the global convergence of an SLP-filter algorithm that takes EQP steps,” Mathematical Programming, vol. 96, no. 1, pp. 161–177, 2003. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  9. M. Ulbrich, S. Ulbrich, and L. N. Vicente, β€œA globally convergent primal-dual interior filter method for nonconvex nonlinear programming,” Mathematical Programming, vol. 100, no. 2, pp. 379–410, 2003. View at: Publisher Site | Google Scholar
  10. R. Fletcher and S. Leyffer, β€œA bundle filter method for nonsmooth nonlinear optimization,” Tech. Rep. NA/195, Department of Mathematics, University of Dundee, Scotland, UK, 1999. View at: Google Scholar
  11. E. Karas, A. Ribeiro, C. Sagastizábal, and M. Solodov, β€œA bundle-filter method for nonsmooth convex constrained optimization,” Mathematical Programming, vol. 116, no. 1-2, pp. 297–320, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  12. N. I. M. Gould, S. Leyffer, and P. L. Toint, β€œA multidimensional filter algorithm for nonlinear equations and nonlinear least-squares,” SIAM Journal on Optimization, vol. 15, no. 1, pp. 17–38, 2004. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  13. N. I. M. Gould, C. Sainvitu, and P. L. Toint, β€œA filter-trust-region method for unconstrained optimization,” SIAM Journal on Optimization, vol. 16, no. 2, pp. 341–357, 2005. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  14. A. Wächter and L. T. Biegler, β€œLine search filter methods for nonlinear programming: motivation and global convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 1–31, 2005. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  15. A. Wächter and L. T. Biegler, β€œLine search filter methods for nonlinear programming: local convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 32–48, 2005. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  16. L. Grippo, F. Lampariello, and S. Lucidi, β€œA nonmonotone line search technique for Newton's method,” SIAM Journal on Numerical Analysis, vol. 23, no. 4, pp. 707–716, 1986. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  17. C. Gu and D. T. Zhu, β€œA non-monotone line search multidimensional filter-SQP method for general nonlinear programming,” Numerical Algorithms, vol. 56, no. 4, pp. 537–559, 2011. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  18. Z. S. Yu and D. G. Pu, β€œA new nonmonotone line search technique for unconstrained optimization,” Journal of Computational and Applied Mathematics, vol. 219, no. 1, pp. 134–144, 2008. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  19. R. Fletcher and S. Leyffer, β€œFilter-type algorithms for solving systems of algebraic equations and inequalities,” Dundee Numerical Analysis Report NA/204, 2001. View at: Google Scholar
  20. P.-Y. Nie, β€œA null space method for solving system of equations,” Applied Mathematics and Computation, vol. 149, no. 1, pp. 215–226, 2004. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  21. P.-Y. Nie, β€œCDT like approaches for the system of nonlinear equations,” Applied Mathematics and Computation, vol. 172, no. 2, pp. 892–902, 2006. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  22. P.-Y. Nie, β€œAn SQP approach with line search for a system of nonlinear equations,” Mathematical and Computer Modelling, vol. 43, no. 3-4, pp. 368–373, 2006. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  23. P.-Y. Nie, M.-Y. Lai, S.-J. Zhu, and P.-A. Zhang, β€œA line search filter approach for the system of nonlinear equations,” Computers & Mathematics with Applications, vol. 55, no. 9, pp. 2134–2141, 2008. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  24. R. H. Byrd, M. Marazzi, and J. Nocedal, β€œOn the convergence of Newton iterations to non-stationary points,” Mathematical Programming, vol. 99, no. 1, pp. 127–148, 2004. View at: Publisher Site | Google Scholar | Zentralblatt MATH

Copyright © 2012 Zhong Jin and Yuqing Wang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder

Related articles

We are committed to sharing findings related to COVID-19 as quickly as possible. We will be providing unlimited waivers of publication charges for accepted research articles as well as case reports and case series related to COVID-19. Review articles are excluded from this waiver policy. Sign up here as a reviewer to help fast-track new submissions.