Global Convergence of a New Nonmonotone Filter Method for Equality Constrained Optimization

Abstract

A new nonmonotone filter trust region method is introduced for solving optimization problems with equality constraints. This method directly uses the dominated area of the filter as an acceptability criterion for trial points and allows the dominated area decreasing nonmonotonically. Compared with the filter-type method, our method has more flexible criteria and can avoid Maratos effect in a certain degree. Under reasonable assumptions, we prove that the given algorithm is globally convergent to a first order stationary point for all possible choices of the starting point. Numerical tests are presented to show the effectiveness of the proposed algorithm.

1. Introduction

We analyze an algorithm for solving optimization problems where a smooth objective function is to be minimized subject to smooth nonlinear equality constraints. More formally, we consider the problem, min𝑓(𝑥),s.t.𝑐_𝑖(𝑥)=0,𝑖∈𝐼={1,2,…,𝑚}, (𝑃) where 𝑥∈𝑅^𝑛, the functions 𝑓∶𝑅^𝑛→𝑅 and 𝑐_𝑖(𝑖∈𝐼)∶𝑅^𝑛→𝑅 are all twice continuously differentiable. For convenience, let 𝑔(𝑥)=∇𝑓(𝑥),𝑐(𝑥)=(𝑐₁(𝑥),𝑐₂(𝑥),…,𝑐_𝑚(𝑥))^𝑇 and 𝐴(𝑥)=(∇𝑐₁(𝑥),∇𝑐₂(𝑥),…,∇𝑐_𝑚(𝑥)), and 𝑓_𝑘 refers to 𝑓(𝑥_𝑘), 𝑐_𝑘 to 𝑐(𝑥_𝑘), 𝑔_𝑘 to 𝑔(𝑥_𝑘) and 𝐴_𝑘 to 𝐴(𝑥_𝑘), and so forth.

There are many trust region methods for equality constrained nonlinear programming (𝑃), for example, Byrd et al. [1], Dennis Jr. et al. [2] and Powell and Yuan [3], but in these works, a penalty or augmented Lagrange function is always used to test the acceptability of the iterates. However, there are several difficulties associated with the use of penalty function, and in particular the choice of the penalty parameter. Hence, in 2002, Fletcher and Leyffer [4] proposed a class of filter method, which does not require any penalty parameter and has promising numerical results. Consequently, filter technique has been employed to many approaches, for instance, SLP methods [5], SQP methods [6–8], interior point approaches [9], bundle techniques [10], and so on.

Filter technique, in fact, exhibits a certain degree of nonmonotonicity. The nonmonotone technique was proposed by Grippo et al. in 1986 [11] and combined with many other methods. M. Ulbrich and S. Ulbrich [12] proposed a class of penalty-function-free nonmonotone trust region methods for nonlinear equality constrained optimization without filter technique. Su and Pu [13] introduced a nonmonotone trust region method which used the nonmonotone technique in the traditional filter criteria. Su and Yu [14] presented a nonmonotone method without penalty function or filter. Gould and Toint [15] directly used the dominated area of the filter as an acceptability criteria for trial points and obtained the global convergence properties. We refer the reader [16–18] for some works about this issue.

Motivated by the ideas and methods above, we propose a modified nonmonotone filter trust region method for solving problem (𝑃). Similar to the Byrd-Omojokun class of algorithms, each step is decomposed into the sum of two distinct components, a quasi-normal step and a tangential step. The main contribution of our paper is to employ the nonmonotone idea to the dominated area of the filter so that the new and more flexible criteria is given, which is different from that of Gould and Toint [15] and Su and Pu [13]. Under usual assumptions, we prove that the given algorithm is globally convergent to first order stationary points.

This paper is organized as follows. In Section 2, we introduce the fraction of Cauchy decrease and the composite SQP step. The new nonmonotone filter technique is given in Section 3. In Section 4, we propose the new nonmonotone filter method and present the global convergence properties in Section 5. Some numerical results are reported in the last section.

2. The Fraction of Cauchy Decrease and the Composite SQP Step

Consider the following unconstraint minimization optimization problem: min_{𝑥∈𝑅𝑛}𝑓(𝑥), (1) where 𝑓∶𝑅^𝑛→𝑅 is a continuously differentiable function. A trust region algorithm for solving the above problem is an iterate procedure that computes a trial step as an approximate solution to the following subproblems: min𝑞(𝑑)=𝑔^𝑇1𝑑+2𝑑^𝑇‖𝑑‖𝐻𝑑,s.t.≤Δ, (2) where 𝐻 is the Hessian matrix ∇²𝑓(𝑥) or an approximate to it and Δ>0 is a given trust region radius.

To assure the global convergence, the step is required only to satisfy a fraction of Cauchy decrease condition. This means that 𝑑 must predict via the quadratic model function 𝑞(𝑑) at least as much as a fraction of the decreased given by the Cauchy step on 𝑞(𝑑); that is, there exists a constant 𝜎>0 fixed across all iterations, such that 𝑞(0)−𝑞(𝑑)≥𝜎𝑞(0)−𝑞𝑑^𝑐𝑝, (3) where 𝑑^𝑐𝑝 is the steepest descent step for 𝑞(𝑑) inside the trust region.

Lemma 1. If the trial step 𝑑 satisfies a fraction of Cauchy decrease condition, then 𝑞(0)(𝑑)≥𝜎−𝑞2(𝑥(𝑥∇𝑓)minΔ,∇𝑓)‖𝐻‖. (4)

Proof (see Powell [19] for the proof). Now, we turn to explain the composite SQP step. Given an approximate estimate of the solution 𝑥_𝑘 at 𝑘th iteration, following Dennis Jr. et al. [2] and M. Ulbrich and S. Ulbrich [12], we obtain the trial step 𝑑_𝑘=𝑑^𝑛_𝑘+𝑑^𝑡_𝑘 by computing a quasi-normal step 𝑑^𝑛_𝑘 and a tangential step 𝑑^𝑡_𝑘. The purpose of the quasi-normal step 𝑑^𝑛_𝑘 is to improve feasibility. To improve optimality, we seek 𝑑^𝑡_𝑘 in the tangential space of the linearized constraints in such a way that it provides sufficient decrease for a quadratic model of the objective function 𝑓(𝑥). Let 𝑞_𝑘(𝑑)=𝑔^𝑇_𝑘𝑑+(1/2)𝑑^𝑇𝐻_𝑘𝑑, where 𝐻_𝑘 is a symmetric approximation of ∇²𝑓(𝑥).
𝑑^𝑛_𝑘 is the solution to the subproblem 1min2𝑐_𝑘+𝐴^𝑇_𝑘𝑑^𝑛2+𝜉2𝑑^𝑛2,s.t.𝑑^𝑛≤Δ_𝑘, (5) where Δ_𝑘 is a trust region radius and 𝐴_𝑘=∇𝑐(𝑥_𝑘)∈𝑅^𝑛×𝑚,𝜉>0. In order to improve the value of the objective function, we solve the following subproblem to get 𝑑^𝑡_𝑘∶min𝑞_𝑘𝑑^𝑛_𝑘+𝑑^𝑡,s.t.𝐴^𝑇_𝑘𝑑^𝑡𝑑=0_𝑡≤Δ_𝑘. (6)
Then we get the current trial step 𝑑_𝑘=𝑑^𝑛_𝑘+𝑑^𝑡_𝑘. Let 𝑑^𝑡_𝑘=𝑊_𝑘𝑑^𝑡_𝑘, where 𝑑^𝑡_𝑘∈𝑅^𝑛−𝑚 and 𝑊_𝑘∈𝑅^{𝑛×(𝑛−𝑚)} denote a matrix whose columns form a basis of the null space of 𝐴^𝑇_𝑘. We refer to [2] for a more detailed discussion of this issue.
In usual way that impose a trust region in step-decomposition methods, the quasi-normal step 𝑑^𝑛_𝑘 and the tangential step 𝑑^𝑡_𝑘 are required to satisfy 𝑑^𝑛≤𝜅Δ_𝑘,𝑑^𝑛_𝑘+𝑑^𝑡≤Δ_𝑘, (7) where 0<𝜅<1. Here, to simplify the proof, we only impose a trust region on ‖𝑑^𝑛‖≤Δ_𝑘 and ‖𝑑^𝑡‖≤Δ_𝑘, which is natural.
Note that 𝑊^𝑇_𝑘∇𝑞_𝑘(𝑑^𝑡) is the reduced gradient of 𝑞_𝑘 in terms of the representation 𝑑^𝑡=𝑊_𝑘𝑠 of the tangential step: ∇_𝑠𝑞_𝑘𝑊_𝑘𝑠=𝑊^𝑇_𝑘∇𝑞_𝑘𝑊_𝑘𝑠=𝑊^𝑇_𝑘∇𝑞_𝑘𝑑^𝑡. (8)
Define 𝑔(𝑥)=𝑊(𝑥)^𝑇𝑔(𝑥). (9)
Then the first order necessary optimality conditions (Karush-Kuhn-Tucker or KKT conditions) at a local solution 𝑥∈𝑅^𝑛 of problem (𝑃) can be written as 𝑐(𝑥)=0,𝑔(𝑥)=0. (10)

3. A New Nonmonotone Filter Technique

In filter method, originally proposed by Fletcher and Leyffer [4], the acceptability of iterates is determined by comparing the value of constraint violation and the objective function with previous iterates collected in a filter. Define the violation function ℎ(𝑥) by ℎ(𝑥)=‖𝑐(𝑥)‖²₂, it is easy to see that ℎ(𝑥)=0 if and only if 𝑥 is a feasible point, so a trial point should reduce either the value of constraint violation or that of the objective function 𝑓.

In the process of the algorithm, we need to decide whether the trial point 𝑥⁺_𝑘 is any better than 𝑥_𝑘 as an approximate solution to the problem (𝑃). If we decide that this is the case, we say that the iteration 𝑘 is successful and choose 𝑥⁺_𝑘 as the next iterate. Let us denote by 𝒮 the set of all successful iterations, that is, 𝒮=𝑘𝑥_𝑘+1=𝑥⁺_𝑘. (11)

In traditional filter method, a point 𝑥 is called acceptable to the filter if and only if ℎ(𝑥)≤𝛽ℎ_𝑗or𝑓(𝑥)≤𝑓_𝑗−𝛾ℎ_𝑗,∀ℎ_𝑗,𝑓_𝑗∈ℱ, (12) where 0<𝛾<𝛽<1, ℱ denotes the filter set. Define 𝒟(ℱ)=ℎ,𝑓ℎ>ℎ_𝑗and𝑓>𝑓_𝑗,∃𝑗∈ℱ. (13)

A trial point 𝑥⁺_𝑘 is accepted if and only if (ℎ⁺_𝑘,𝑓⁺_𝑘)∉𝒟(ℱ_𝑘).

Now, similar to the idea of Gould and Toint [15], we give a new modified nonmonotone filter technique. For any (ℎ,𝑓)-pair, define an area that represents its contribution to the area of 𝒟(ℱ), we hope this contribution is positive; that is, the area of 𝒟(ℱ) is increasing. For convenience, we partition the right half-plane [0,+∞]×[−∞,+∞] into four different regions (see Figure 1). Define 𝒟(ℱ_𝑘)^𝑐 to be the complement of 𝒟(ℱ_𝑘). Let ℎ=^ℱ𝑘_min^defmin_{𝑗∈ℱ𝑘}ℎ_𝑗,ℎ=^ℱ𝑘_max^defmax_{𝑗∈ℱ𝑘}ℎ_𝑗,𝑓=^ℱ𝑘_min^defmin_{𝑗∈ℱ𝑘}𝑓_𝑗,𝑓=^ℱ𝑘_max^defmax_{𝑗∈ℱ𝑘}𝑓_𝑗. (14)

Figure 1 

These four parts are(1)the dominated part of the filter: 𝑁𝐸(ℱ_𝑘)=^def𝒟(ℱ_𝑘);(2)the undominated part of lower left corner of the half plane: 𝑆𝑊ℱ=_𝑘^def𝒟ℱ_𝑘^𝑐∩0,ℎ^ℱ𝑘_max×−∞,𝑓^ℱ𝑘_max, (15)(3)the undominated upper left corner: 𝑁𝑊(ℱ_𝑘)=^def[0,ℎ^ℱ𝑘_min)×(𝑓^ℱ𝑘_max,+∞];(4)the undominated lower right corner: 𝑆𝐸(ℱ_𝑘)=^def(ℎ^ℱ𝑘_max,+∞]×[−∞,𝑓^ℱ𝑘_min).

Consider the trial point 𝑥⁺_𝑘, if the filter is empty, then define its contribution to the area of the filter by 𝛼𝑥⁺_𝑘,ℱ=_𝑘^def𝜅_𝐹², (16) where 𝜅_𝐹>0 is a constant. If the filter is not empty, then define the contribution of 𝑥⁺_𝑘 to the area of the filter by four different formulae.

If (ℎ⁺_𝑘,𝑓⁺_𝑘)∈𝑆𝑊(ℱ_𝑘), assume 𝛼𝑥⁺_𝑘,ℱ=_𝑘^defarea𝒟ℱ_𝑘^𝑐∩ℎ⁺_𝑘,ℎ^ℱ𝑘_max+𝜅_𝐹×𝑓⁺_𝑘,𝑓^ℱ𝑘_max+𝜅_𝐹. (17)

If (ℎ⁺_𝑘,𝑓⁺_𝑘)∈𝑁𝑊(ℱ_𝑘), assume 𝛼𝑥⁺_𝑘,ℱ=_𝑘^def𝜅_𝐹ℎ^ℱ𝑘_min−ℎ⁺_𝑘. (18)

If (ℎ⁺_𝑘,𝑓⁺_𝑘)∈𝑆𝐸(ℱ_𝑘), assume 𝛼𝑥⁺_𝑘,ℱ=_𝑘^def𝜅_𝐹𝑓^ℱ𝑘_min−𝑓⁺_𝑘. (19)

If (ℎ⁺_𝑘,𝑓⁺_𝑘)∈𝑁𝐸(ℱ_𝑘)=𝒟(ℱ_𝑘), assume 𝛼𝑥⁺_𝑘,ℱ=_𝑘^def−area𝒟ℱ_𝑘∩ℎ⁺_𝑘−ℎ^𝒫𝑘_min×𝑓⁺_𝑘−𝑓^𝒫𝑘_min, (20) where 𝒫_𝑘={(ℎ,𝑓)∈ℱ_𝑘|ℎ_𝑗<ℎ⁺_𝑘,𝑓_𝑗<𝑓⁺_𝑘}, and ℎ=^𝒫𝑘_min^defmin_{𝑗∈𝒫𝑘}ℎ_𝑗,𝑓=^𝒫𝑘_min^defmin_{𝑗∈𝒫𝑘}𝑓_𝑗.

Figure 2 illustrate the corresponding areas in the filter. Horizontally dashed surfaces indicate a positive contribution and vertically dashed ones a negative contribution. Note that 𝛼(𝑥,ℱ) is a continuous function of (ℎ(𝑥),𝑓(𝑥)).

Figure 2 

Next, we should consider the updating of the filter. If (ℎ_𝑘,𝑓_𝑘)∉𝒟(ℱ_𝑘), then ℱ_𝑘+1⟵ℱ_𝑘∪ℎ_𝑘,𝑓_𝑘. (21)

If (ℎ_𝑘,𝑓_𝑘)∈𝒟(ℱ_𝑘), then ℱ_𝑘+1⟵ℱ_𝑘⧵𝒫_𝑘∪ℎ^𝒫𝑘_min,𝑓_𝑘∪ℎ_𝑘,𝑓^𝒫𝑘_min. (22)

We now return to the question of deciding whether a trial point 𝑥⁺_𝑘 is acceptable for the filter or not. We will insist that this is a necessary condition for the iteration 𝑘 to be successful in the sense that 𝑥_𝑘+1=𝑥⁺_𝑘. If we consider an iterate 𝑥_𝑘, there must exist a predecessor iteration such that 𝑥⁺_𝑝(𝑘)=𝑥_𝑝(𝑘)+1=𝑥_𝑘. Under the monotonic situation, a trial point 𝑥⁺_𝑘 would be accepted whenever it results in an sufficient increase in the dominated area of the filter, that means 𝑥⁺_𝑘 would be accepted whenever 𝛼_𝑘≥𝛾_ℱℎ⁺_𝑘², (23) where 𝛼=_𝑘^def𝛼(𝑥⁺_𝑘,ℱ_𝑘), 𝛾_ℱ∈(0,1) is a constant. Under the nonmonotonic situation, we relax condition (23) to be max𝛼_𝑘,^𝑘_{𝑗=𝑟(𝑘)+1𝑗∈𝒰}𝜆_𝑝(𝑗)𝛼_𝑝(𝑗)≥𝛾_ℱℎ⁺_𝑘², (24) where 𝛼=_𝑝(𝑗)^def𝛼(𝑥_𝑗,ℱ_𝑝(𝑗)),𝒰={𝑘|filterisupdatedfor(ℎ_𝑘,𝑓_𝑘)}, 𝑟(𝑘)≤𝑘 is some reference iteration for 𝒰, 𝑟(𝑘)∈𝒰, 𝒰⊆𝒮, 𝜆_𝑝(𝑗)∈[0,1],∑^𝑘_{𝑗=𝑟(𝑘)+1,𝑗∈𝒰}𝜆_𝑝(𝑗)=1.

Compared to condition (2.21) [15], our condition (24) is more flexible if 𝛼_𝑘 is negative.

According to condition (24), it is possible to accept 𝑥⁺_𝑘 even though it may be dominated. Then 𝑥⁺_𝑘 will be accepted if either (23) or (24) holds.

4. The New Nonmonotone Filter Trust Region Algorithm

Our algorithm is based on the usual trust region technique; define the predict reduction for the function 𝑞_𝑘(𝑥) to be pred𝑑_𝑘=𝑞_𝑘(0)−𝑞_𝑘𝑑_𝑘 (25) and the actual reduction ared𝑑_𝑘=𝑓𝑥_𝑘−𝑓𝑥⁺_𝑘. (26)

Moreover, let 𝑟_𝑘=ared(𝑑_𝑘)/pred(𝑑_𝑘), if there exists a nonzero constant 𝜂₁ such that 𝑟_𝑘≥𝜂₁ and condition (23) and (24) hold, the trial point 𝑥⁺_𝑘 will be called acceptable. Then the next trial point 𝑥_𝑘+1 is obtained, and for its feasibility, we consider the condition ℎ_𝑘+1≤𝜂₃min𝜇,𝛼₁Δ^2+𝛼2_𝑘 (27) is true or not, where 𝜂₃ and 𝜇 are all positive constants, if it is not true, then turn to the feasibility restoration phase and define 𝑟^𝑗_𝑘=𝑐(𝑥^𝑗_𝑘)²−𝑐(𝑥^𝑗_𝑘+𝑑^𝑗_𝑘)²𝑐(𝑥^𝑗_𝑘)²−𝑐(𝑥^𝑗_𝑘)+𝐴(𝑥^𝑗_𝑘)^𝑇𝑑^𝑗_𝑘². (28) A formal description of the algorithm is given as follows.

Algorithm A 𝑆𝑡𝑒𝑝0. Choose an initial point 𝑥₀∈𝑅^𝑛, a symmetric matrix 𝐻₀∈𝑅^𝑛×𝑛, let Δ₀>0, 𝜖_𝑡>0,𝜂₂,𝜂₃∈(0,1),𝛼₁,𝛼₂∈[0,1],𝜂₁>0,0<𝛾₀<𝛾₁<1≤𝛾₂<𝛾₃≤2,0<𝜉<1,𝛾_ℱ∈(0,1),𝜇>0, compute 𝑓(𝑥₀),ℎ(𝑥₀), let 𝑘=0,ℱ₀=∅.

Step 1. Compute 𝑐_𝑘,𝐴_𝑘,𝑊_𝑘,𝑓_𝑘,ℎ_𝑘,𝑔_𝑘,𝑔_𝑘=𝑊^𝑇_𝑘𝑔_𝑘.

Step 2. If ‖𝑔_𝑘‖+ℎ_𝑘≤𝜖_𝑡, stop.

Step 3. Solve the subproblems (5) and (6) to get the quasi-normal step 𝑑^𝑛_𝑘 and the tangential step 𝑑^𝑡_𝑘. Let 𝑑_𝑘=𝑑^𝑛_𝑘+𝑑^𝑡_𝑘,𝑥⁺_𝑘=𝑥_𝑘+𝑑_𝑘.

Step 4. If 𝑟_𝑘<𝜂₁, let 𝑥_𝑘+1=𝑥_𝑘, then go to Step 8.

Step 5. If 𝑥_𝑘+𝑑_𝑘 is not acceptable to the filter, go to Step 8. Otherwise 𝑥_𝑘+1=𝑥⁺_𝑘 and update the filter according to (21) and (22), the trust region radius and 𝐻_𝑘, then get the corresponding ℎ_𝑘+1,𝑓_𝑘+1.

Step 6. If ℎ_𝑘+1≤𝜂₃min{𝜇,𝛼₁Δ^2+𝛼2_𝑘}, 𝑘=𝑘+1 and go to Step 1, otherwise go to Step 7.

Step 7. By restoration Algorithm B to get 𝑑^𝑟_𝑘, then the trial point 𝑥^𝑟_𝑘=𝑥_𝑘+𝑑^𝑟_𝑘.

Step 8. Update the trust region radius by Algorithm C, let 𝑘=𝑘+1 and go to Step 3.
 We aim to reduce the value of ℎ(𝑥) in the restoration Algorithm B, that is to get 𝑐(𝑥^𝑟_𝑘)=0 by Newton-type method.

Algorithm B 𝑆𝑡𝑒𝑝0. Let 𝑥⁰_𝑘=𝑥_𝑘,Δ⁰_𝑘=Δ_𝑘,𝑗=0,𝜂₂,𝜂₃∈(0,1),𝛼₁,𝛼₂∈[0,1],𝜇>0.

Step 1. If ℎ(𝑥^𝑗_𝑘)≤𝜂₃min{ℎ^ℱ𝑘_𝑘,𝛼₁Δ^2+𝛼2_𝑘} and 𝑥^𝑗_𝑘 is acceptable by the filter, then 𝑥^𝑟_𝑘=𝑥^𝑗_𝑘, stop.

Step 2. Compute min𝑐(𝑥^𝑗_𝑘)+𝐴(𝑥^𝑗_𝑘)^𝑇𝑑^𝑗_𝑘²,s.t.𝑑^𝑗_𝑘≤Δ^𝑗_𝑘 (29) to get 𝑑^𝑗_𝑘, then compute 𝑟^𝑗_𝑘.

Step 3. If 𝑟^𝑗_𝑘≤𝜂₂, 𝑥_𝑘^𝑗+1=𝑥^𝑗_𝑘,Δ_𝑘^𝑗+1=Δ^𝑗_𝑘/2,𝑗=𝑗+1 and go to Step 2.

Step 4. If 𝑟^𝑗_𝑘>𝜂₂, 𝑥_𝑘^𝑗+1=𝑥^𝑗_𝑘+𝑑^𝑗_𝑘,Δ_𝑘^𝑗+1=2Δ^𝑗_𝑘,𝑗=𝑗+1, compute 𝐴_𝑘^𝑗+1 and go to Step 1, where ℎ^ℱ𝑘_𝑘=min{min_{𝑗∈ℱ𝑘}{ℎ_𝑗|ℎ_𝑗>0},𝜇}.

Algorithm C (updating the trust region radius). Given 𝜂₁>0,0<𝛾₀<𝛾₁<1≤𝛾₂<𝛾₃≤2, we have the following.(1)If 𝑟_𝑘<𝜂₁ or 𝑥⁺_𝑘 is not acceptable to the filter, Δ_𝑘+1∈[𝛾₀Δ_𝑘,𝛾₁Δ_𝑘].(2)If 𝑥⁺_𝑘 is acceptable to the filter but does not satisfiy condition (27), Δ_𝑘+1∈(Δ_𝑘,𝛾₂Δ_𝑘).(3)If 𝑥⁺_𝑘 is acceptable to the filter and satisfies (27),  Δ_𝑘+1∈[𝛾₂Δ_𝑘,𝛾₃Δ_𝑘].

From the description above and the idea of the algorithm, we can see that our algorithm is more flexible. Every successful iterate must be any better than the predecessor one in some degree according to the traditional filter method. But our algorithm relaxes this demand by using the nonmonotone technique and also avoids Maratos effect in a certain degree. Moreover, Algorithm C allows a relatively wide choice of the trust region.

5. The Convergence Properties

In this section, to present a proof of global convergence of algorithm, we always assume that the following conditions hold.

Assumption

(A1) The objective function 𝑓 and the constraint functions 𝑐_𝑖(𝑖∈𝐼={1,2,…,𝑚}) are twice continuously differentiable. (A2) For all 𝑘,  𝑥_𝑘, and 𝑥_𝑘+𝑑_𝑘 all remain in a closed, bounded convex subset Ω⊂𝑅^𝑛. (A3) The matrix 𝐴(𝑥)=∇𝑐(𝑥) is nonsingular matrix for all 𝑥∈Ω. (A4) The matrices (𝐴(𝑥)^𝑇𝐴(𝑥))⁻¹,𝑊(𝑥),(𝑊^𝑇𝑊)⁻¹ are uniformly bounded in Ω, where 𝑊(𝑥) denotes a matrix whose columns form a basis of the null space of 𝐴(𝑥)^𝑇. (A5) The matrix 𝐻_𝑘 is uniformly bounded.

By the assumptions, we can suppose there exist constants 𝑣₀,𝑣₁,𝑣₂,𝑣₃ such that ‖𝑓(𝑥)‖≤𝑣₀,‖𝑔(𝑥)‖≤𝑣₀,‖𝑐(𝑥)‖≤𝑣₀,‖𝐴(𝑥)‖≤𝑣₀,‖𝐴(𝑥)^𝑇𝐴(𝑥)⁻¹‖≤𝑣₁,‖𝑊(𝑥)‖≤𝑣₂,‖𝐻_𝑘𝑣‖≤₃,‖𝑊^𝑇_𝑘𝐻_𝑘‖≤𝑣₃,‖𝑊^𝑇_𝑘𝐻_𝑘𝑊_𝑘‖≤𝑣₃.

By (A1) and (A2), it holds 𝑓_min≤𝑓_𝑘,0≤ℎ_𝑘≤ℎ_max∀𝑘, (30) where 𝑓_min,ℎ_max>0, hence in the (ℎ,𝑓)-plane, the (ℎ,𝑓)-pair lies in the area [0,ℎ_max]×[𝑓_min,+∞].

From (A1), (A2), and (A3), it exists a constant 𝜈 such that 𝑓𝑥_𝑘+𝑑_𝑘−𝑞_𝑘𝑑_𝑘≤𝜈Δ²_𝑘. (31)

Lemma 2. At the current iterate 𝑥_𝑘, let the trial point component 𝑑^𝑛_𝑘 actually be normal to the tangential space. Under the problem assumptions, there exists a constant 𝑘₁>0 independent of the iterates such that 𝑑^𝑛_𝑘≤𝛼₁𝑐_𝑘. (32)

Proof. It is similar to the proof of Lemma 2 in [13].

Lemma 3. Under Assumptions, there exist positive constants 𝑘₂,𝑘₃,𝑘₄ independent of the iterates such that 𝑐_𝑘²−𝑐_𝑘+𝐴^𝑇_𝑘𝑑^𝑛_𝑘²≥𝑘₂𝑐_𝑘min𝑘₃𝑐_𝑘,Δ_𝑘,𝑞_𝑘𝑑^𝑛_𝑘−𝑞_𝑘𝑑_𝑘≥𝜎2𝑊^𝑇_𝑘∇𝑞_𝑘𝑑^𝑛_𝑘min𝑘₄𝑊^𝑇_𝑘∇𝑞_𝑘𝑑^𝑛_𝑘,Δ_𝑘. (33)

Proof. The proof is an application of Lemma 1 to the two subproblems (5) and (6).

Lemma 4. Suppose that Assumptions hold, then restoration Algorithm B is well defined.

Proof. The conclusion is obvious, if ℎ^𝑗_𝑘→0. Otherwise it exists 𝜖>0 such that for all 𝑗, it holds ℎ^𝑗_𝑘>𝜖. Consider the set 𝐾=𝑗𝑟^𝑗_𝑘=𝑐^𝑗_𝑘²−𝑐𝑥^𝑗_𝑘+𝑑^𝑗_𝑘²𝑐^𝑗_𝑘²−𝑐^𝑗_𝑘+(𝐴^𝑗_𝑘)^𝑇𝑑^𝑗_𝑘²>𝜂₂>0, (34) where 𝑐^𝑗_𝑘=𝑐(𝑥^𝑗_𝑘),𝐴^𝑗_𝑘=𝐴(𝑥^𝑗_𝑘). By Lemma 3 and the definition of ℎ_𝑘, we have +∞>^∞_𝑗=1ℎ_𝑘^𝑗−1−ℎ^𝑗_𝑘≥_𝑗∈𝐾𝑐^𝑗_𝑘²−𝑐^𝑗_𝑘+𝐴^𝑗_𝑘^𝑇𝑑^𝑗_𝑘²≥𝜂₂𝑘_2𝑗∈𝐾𝑐_𝑘min𝑘₃𝑐_𝑘,Δ^𝑗_𝑘. (35)
By ℎ^𝑗_𝑘>𝜖, it holds Δ^𝑗_𝑘→0 for 𝑗∈𝐾. From Algorithm B, we can obtain that Δ^𝑗_𝑘→0 for all 𝑗.
On the other side, 𝑐^𝑗_𝑘²−𝑐𝑥^𝑗_𝑘+𝑑^𝑗_𝑘²=𝑐^𝑗_𝑘²−𝑐^𝑗_𝑘+𝐴^𝑗_𝑘^𝑇𝑑^𝑗_𝑘²+𝑜Δ^𝑗_𝑘 (36) for Δ^𝑗_𝑘→0. By the algorithm, the radius Δ^𝑗_𝑘 should be satisfied Δ_𝑘^𝑗+1>Δ^𝑗_𝑘, that is contradicted to Δ^𝑗_𝑘→0. The proof is complete.

Now, we analyze the impact of the criteria (23) and (24). Once a trial point is accepted as a new iterate, it must be provided some improvement, and we formalize this by saying that iterate 𝑥_𝑘=𝑥_𝑝(𝑘)+1 improves on iterate 𝑥_𝑖(𝑘). That is the trial point 𝑥_𝑘 is accepted at iterate 𝑝(𝑘); it happens under two situations, one is by the criteria (23), that is, 𝑖(𝑘)=𝑝(𝑘)if𝑝(𝑘)∉𝒜 (37) the other is by the criteria (24), that is, 𝑖(𝑘)=𝑝(𝑘)if𝑝(𝑘)∉𝒜. (38)

Now consider any iterate 𝑥_𝑘, it improved on 𝑥_𝑖(𝑘), which was itself accepted because it improved on 𝑥_{𝑖(𝑖(𝑘))}, and so on, until back to 𝑥₀. Hence we may construct a chain of successful iterations indexed by 𝒞_𝑘= {𝑙₁,𝑙₂,…,𝑙_𝑞} for each 𝑘, such that 𝑥_𝑙1=𝑥₀,𝑥_𝑙𝑞=𝑥_𝑘,𝑥_𝑙𝑗=𝑥_{𝑖(𝑙𝑗+1)},𝑗=1,2,…,𝑞−1, (39) where 𝑙₁ is the smallest index in the chain of successful iterations.

Lemma 5. Suppose that Assumptions hold and Algorithm A does not terminate finitely, apply Algorithm A to the problem (𝑃), then for all 𝑘 and 𝒞_𝑘={𝑙₁,𝑙₂,…,𝑙_𝑞}, it holds area𝒟ℱ_𝑘≥𝛾_ℱ^𝑞_𝑗=1ℎ²_𝑙𝑗. (40)

Proof. For ∀𝑙_𝑗∈𝒞_𝑘, if 𝑝(𝑙_𝑗)∈𝒜, by (24),  𝑖(𝑙_𝑗)=𝑟(𝑝(𝑙_𝑗))=𝑙_𝑗−1, then max𝛼_{𝑝(𝑙𝑗)},^𝑙𝑗_{𝑖=𝑙𝑗−1+1𝑖∈𝒰}𝜆_𝑝(𝑖)𝛼_𝑝(𝑖)≥𝛾_ℱℎ²_𝑙𝑗. (41)
If max{𝛼_{𝑝(𝑙𝑗)},∑^𝑙𝑗_{𝑖=𝑙𝑗−1+1,𝑖∈𝒰}𝜆_𝑝(𝑖)𝛼_𝑝(𝑖)}=𝛼_{𝑝(𝑙𝑗)}, 𝛼_{𝑝(𝑙𝑗)}≥𝛾_ℱℎ²_𝑙𝑗. (42)
If max{𝛼_{𝑝(𝑙𝑗)},∑^𝑙𝑗_{𝑖=𝑙𝑗−1+1,𝑖∈𝒰}𝜆_𝑝(𝑖)𝛼_𝑝(𝑖)}=∑^𝑙𝑗_{𝑖=𝑙𝑗−1+1,𝑖∈𝒰}𝜆_𝑝(𝑖)𝛼_𝑝(𝑖), ^𝑙𝑗_{𝑖=𝑙𝑗−1+1𝑖∈𝒰}𝛼_𝑝(𝑖)≥𝛾_ℱℎ²_𝑙𝑗. (43)
If 𝑝(𝑙_𝑗)∉𝒜,  𝑙_𝑗−1=𝑝(𝑙_𝑗). By 𝒰⊆𝒮, it holds 𝑙_𝑗−1+1,…,𝑙_𝑗∩𝒰⊆𝑙_𝑗−1+1,…,𝑙_𝑗∩𝒮=𝑙_𝑗. (44)
Then from (23), 𝛼_𝑝(𝑖)≥𝛾_ℱℎ²_𝑙𝑗. It implies (43). Moreover area𝒟ℱ_𝑘≥^𝑘_{𝑖=0𝑖∈𝒰}𝛼_𝑝(𝑖)=^𝑞_𝑗=0^𝑙𝑗_{𝑖=𝑙𝑗−1+1𝑖∈𝒰}𝛼_𝑝(𝑖). (45)
Together with (42) and (43), the result follows.

Lemma 6. Suppose that Assumptions hold. If Algorithm A does not terminate finitely and the filter contains infinite iterates, then lim_𝑘→∞ℎ_𝑘=0.

Proof. Suppose by contradiction that there exists a constant 𝜖>0 and infinite sequence {𝑘_𝑖}⊆𝒮 such that ℎ_𝑘𝑖≥𝜖 for all 𝑖. Because there are infinite iterations in the filter, we have |𝒮|=∞, then ℎ_𝑙𝑞≥𝜖 for ∀𝑞. area𝒟ℱ_𝑘≥𝛾_ℱ⋅𝑞⋅𝜖². (46) Then by (31), area(𝒟(ℱ_𝑘)) is upper bounded for each 𝑘. That means it exists 𝜅_ℱ^max≥0 such that area(𝒟(ℱ_𝑘))≤𝜅_ℱ^max, so 𝑖≤(𝜅_ℱ^max/𝛾_ℱ𝜖²). Hence 𝑖 must be finite, it contradicts to the infinity of {𝑘_𝑖}. The proof is complete.

Lemma 7. Suppose that Assumptions hold and Algorithm A terminate finitely, then ℎ_𝑘=0.

Proof. From the Algorithm A and the definition of filter, the conclusion follows.

Lemma 8. For any trial point 𝑥_𝑘+1≠𝑥_𝑘, there must be one accepted by the filter.

Lemma 9. Suppose that Assumptions hold, there exists 𝑘₅>0 independent of the iterates such that 𝑞_𝑘(0)−𝑞_𝑘𝑑^𝑛_𝑘≥−𝑘₅𝑐_𝑘. (47)

Proof. By (32), the assumptions and ‖𝑑^𝑛_𝑘‖≤Δ_max, it is obvious that 𝑞_𝑘(0)−𝑞_𝑘𝑑^𝑛_𝑘=−𝑔^𝑇_𝑘𝑑^𝑛_𝑘−12𝑑^𝑛_𝑘^𝑇𝐻_𝑘𝑑^𝑛_𝑘≥−𝑑^𝑛_𝑘𝑔_𝑘+12𝐻_𝑘𝑑^𝑛_𝑘≥−𝑘₁𝑐_𝑘𝜈₀+𝜈₃Δ_maxdef=−𝑘₅𝑐_𝑘. (48) The proof is complete.

Lemma 10. Suppose that Assumptions hold and ‖𝑔_𝑘‖≥𝜖_𝑡, if Δ_𝑘𝜖≤min^𝑡2𝑘₁𝜈₃√𝜂₃𝛼₁^2/(2+𝛼2),𝑘₄𝜖_𝑡2,𝜈₀𝑘₁𝜈₃√𝜂₃𝛼₁^2/(2+𝛼2),𝜎𝜖_𝑡16𝑘₁𝜈₀√𝜂₃𝛼₁^2/𝛼2def=𝛿₁ (49) one can deduce 𝑞_𝑘(0)−𝑞_𝑘𝑑_𝑘≥𝜎𝜖_𝑡8Δ=_𝑘^def𝜎Δ_𝑘. (50)

Proof. By the assumptions and the definition of 𝑑^𝑛_𝑘, it holds 𝑊^𝑇_𝑘∇𝑞_𝑘𝑑^𝑛_𝑘=𝑊^𝑇_𝑘𝑔_𝑘+𝐻_𝑘𝑑^𝑛_𝑘≥𝑔_𝑘−𝑊^𝑇_𝑘𝐻_𝑘𝑑^𝑛_𝑘≥𝜖_𝑡−𝜈₃𝑘₁𝑐_𝑘≥𝜖_𝑡−𝜈₃𝑘₁√𝜂₃𝛼₁Δ^1+(𝛼2_𝑘^/2)≥𝜖_𝑡2. (51) From Lemma 3, 𝑞_𝑘(𝑑^𝑛_𝑘)−𝑞_𝑘(𝑑_𝑘)≥(𝜎/4)𝜖_𝑡min{Δ_𝑘,𝑘₄𝜖_𝑡/2}≥(𝜎𝜖_𝑡/4)Δ_𝑘. Together with 𝑞_𝑘(0)−𝑞_𝑘𝑑^𝑛_𝑘=−𝑔^𝑇_𝑘𝑑^𝑛_𝑘−12𝑑^𝑛_𝑘^𝑇𝐻_𝑘𝑑^𝑛_𝑘≥−𝑔_𝑘𝑑^𝑛_𝑘−12𝐻_𝑘𝑑^𝑛_𝑘²≥−𝑘₁𝜈₀√𝜂₃𝛼₁Δ^1+(𝛼2_𝑘^/2)−𝑘²₁𝜈₃𝜂₃𝛼₁Δ^2+𝛼2_𝑘≥−2𝑘₁𝜈₀√𝜂₃𝛼₁Δ^1+(𝛼2_𝑘^/2)≥−𝜎𝜖_𝑡8Δ_𝑘 (52) then 𝑞_𝑘(0)−𝑞_𝑘(𝑑_𝑘)≥−(𝜎𝜖_𝑡/8)Δ_𝑘+(𝜎𝜖_𝑡/4)Δ_𝑘=(𝜎𝜖_𝑡/8)Δ_𝑘=𝜎Δ_𝑘. It is the conclusion.

Lemma 11. Suppose the conditions of Lemma 10 hold, if Δ_𝑘≤min𝛿₁,1−𝜂₁𝜎𝜈^def=𝛿₂ (53) then 𝑟_𝑘≥𝜂₁.

Proof. From the definition of 𝑟_𝑘 and Lemma 10, together with (31), we have 𝑟_𝑘=𝑓𝑥−1^𝑘+𝑑_𝑘−𝑞_𝑘𝑑_𝑘𝑞_𝑘(0)−𝑞_𝑘𝑑_𝑘≤𝜈Δ²_𝑘𝜎Δ_𝑘≤1−𝜂₁. (54) It is obvious that 𝑟_𝑘≥𝜂₁.

Lemma 12. Suppose the conditions of Lemmas 10 and 11 hold, if ℎ_𝑘≤𝜂₃𝛼₁^{−1/(1+𝛼2)}𝜂₁𝜎√𝛾_ℱ^{(2+𝛼2)/(1+𝛼2)} (55) then 𝑓𝑥⁺_𝑘≤𝑓𝑥_𝑘−√𝛾_ℱℎ_𝑘. (56)

Proof. By Lemmas 3, 10, and 11, together with ‖𝑔_𝑘‖≥𝜖_𝑡, it holds 𝑓𝑥_𝑘−𝑓𝑥⁺_𝑘≥𝜂₁𝑞_𝑘(0)−𝑞_𝑘𝑑_𝑘≥𝜂₁𝜎Δ_𝑘. (57) From the Algorithm, ℎ_𝑘≤𝜂₃𝛼₁Δ^2+𝛼2_𝑘, then 𝑓𝑥_𝑘−𝑓𝑥⁺_𝑘≥𝜂₁𝜎ℎ_𝑘𝜂₃𝛼₁^1/(2+𝛼2). (58) Hence 𝑓(𝑥_𝑘)−𝑓(𝑥⁺_𝑘)≥√𝛾_ℱℎ_𝑘.

Theorem 13. Suppose the assumptions hold, there must exist Δ_min>0 such that for each 𝑘, it holds Δ_𝑘≥Δ_min. (59)

Proof. Let 𝑘₁ be large enough such that ℎ_𝑘1≤𝜖_𝑡, it is true by Lemmas 6 and 7. Suppose by contradiction that the index 𝑗 is the first one after 𝑘₁, which satisfies Δ_𝑗≤𝛾₀min𝛿₂,1−√𝛾_ℱℎ^𝐹𝜂₃𝛼₁^1/(2+𝛼2),Δ=_𝑘1^def𝛾₀𝛿₃, (60) where ℎ=^𝐹defmin_𝑖∈𝒰ℎ_𝑖 is the smallest value of violation function in filter. Then Δ_𝑗≤𝛾₀Δ_𝑘1. By the above analysis, we know 𝑗≥𝑘₁+1, that is 𝑗−1≥𝑘₁. From the Algorithm and (60), it concludes Δ_𝑗−1≤1𝛾₀Δ_𝑗≤𝛿₃. (61)
By (60) and (61), (53) can be obtained. In Lemma 11, let 𝑗−1 instead of 𝑘, it deduces 𝑟_𝑗−1≥𝜂₁. (62)
Based on Lemma 12, together with (60), (61), and the algorithm, we can see ℎ⁺_𝑗−1≤𝜂₃𝛼₁Δ^2+𝛼2_𝑗−1≤1−√𝛾_ℱℎ^𝐹. (63)
It can be seen that (53) is true for 𝑗−1≥𝑘, with (55), we can deduce 𝑓⁺_𝑗−1≤𝑓_𝑗−1−√𝛾_ℱℎ_𝑗−1. (64)
That means 𝑥⁺_𝑗−1 can be accepted by the filter. From above and (55), we know Δ_𝑗≥Δ_𝑗−1. Hence the index 𝑗 is not the first one after 𝑘₁ which satisfied (60), that is a contradiction. So, for any 𝑘>𝑘₁, it holds Δ_𝑘≥𝛾₀𝛿₃. Define Δ_min=minΔ₀,…,Δ_𝑘1,𝛾₀𝛿₃ (65)
we can see that Δ_𝑘≥Δ_min (66) holds for each 𝑘. The proof is complete.

Lemma 14. Suppose that Assumptions hold and Algorithm A does not terminate finitely, then liminf_𝑘→∞‖𝑔_𝑘‖=0.

Proof. Suppose by contradiction that for 𝜖_𝑡, there exists a constant 𝑘>0 such that ‖𝑔_𝑘‖≥𝜖_𝑡.
By Assumption (A3) and (A4), ‖𝑊^𝑇_𝑘∇𝑞_𝑘(𝑑^𝑛_𝑘)‖≥‖𝑔_𝑘‖−𝜈₃𝑘₁‖𝑐_𝑘‖. From Lemma 6, we know ℎ_𝑘→0. Hence there exists 𝑘>0 such that 𝑐_𝑘≤2𝜖_𝑡3𝜈₃𝑘₁ (67) for 𝑘>𝑘. Then ‖𝑊^𝑇_𝑘∇𝑞_𝑘(𝑑^𝑛_𝑘)‖≥(1/3)‖𝑔_𝑘‖≥(1/3)𝜖_𝑡 for 𝑘>𝑘=^defmax{𝑘,𝑘}.
It is obvious that 𝑞_𝑘(0)−𝑞_𝑘𝑑_𝑘=𝑞_𝑘(0)−𝑞_𝑘𝑑^𝑛_𝑘+𝑞_𝑘𝑑^𝑛_𝑘−𝑞_𝑘𝑑_𝑘. (68)
By the proof of Lemma 9, it holds |𝑞_𝑘(0)−𝑞_𝑘(𝑑^𝑛_𝑘)|≤𝜈₀‖𝑑^𝑛_𝑘‖+(1/2)𝜈₃‖𝑑^𝑛_𝑘‖². Together with Lemma 6 and the definition of 𝑑_𝑘, we have lim_𝑘→∞𝑞_𝑘(0)−𝑞_𝑘𝑑^𝑛_𝑘=0. (69) By the Algorithm, we can get +∞>^∞_𝑘=0𝑓_𝑘−𝑓_𝑘+1≥𝜂₁^∞_𝑘=0𝑞_𝑘(0)−𝑞_𝑘𝑑_𝑘. (70) Then lim_𝑘→∞𝑞_𝑘(0)−𝑞_𝑘𝑑_𝑘=0. (71) By (68), (69), and (71), it deduces lim_𝑘→∞𝑞_𝑘𝑑^𝑛_𝑘−𝑞_𝑘𝑑_𝑘=0. (72) Based on the assumptions, Lemma 3 and Theorem 13, for 𝑘>𝑘, it holds 𝑞_𝑘𝑑^𝑛_𝑘−𝑞_𝑘𝑑_𝑘≥𝜎2𝑊^𝑇_𝑘∇𝑞_𝑘𝑑^𝑛_𝑘minΔ_𝑘,𝑘₄𝑊^𝑇_𝑘∇𝑞_𝑘𝑑^𝑛_𝑘≥𝜎𝜖_𝑡6minΔ_min,𝑘₄𝜖_𝑡6>0, (73) which contradicts (72). The conclusion follows.

Theorem 15. Suppose the assumptions hold, and apply the algorithm to problem (𝑃), then liminf_𝑘→∞ℎ_𝑘+𝑔_𝑘=0, (74) where 𝑔_𝑘=𝑊^𝑇_𝑘𝑔_𝑘,𝑔_𝑘=∇𝑓(𝑥_𝑘),  𝑊(𝑥) denotes a matrix whose columns form a basis of the null space of 𝐴(𝑥)^𝑇.

Proof. If the algorithm terminates finitely, it is obvious that it holds. Otherwise, by Lemmas 6 and 14, the conclusion also can be obtained.

Theorem 16. Suppose the assumptions hold, and {𝑥_𝑘} is the infinite sequence obtained by the algorithm, then there must exist a subsequence such that lim_𝑗→∞𝑥_𝑘𝑗=𝑥^∗ (75) and 𝑥^∗ satisfies the one order KKT condition of (𝑃).

Proof. By Assumption (A1), there exist a subsequence {𝑘_𝑗} and 𝑥^∗, such that lim_𝑗→∞𝑥_𝑘𝑗=𝑥^∗. Together with Assumption (A3) and (A4), it hods lim_𝑗→∞𝑊^𝑇_𝑘𝑗𝑔_𝑘𝑗=0, which means for large enough 𝑗, 𝑔_𝑘𝑗 lies in the space spaned by the columns of 𝐴^𝑇_𝑘𝑗. That is there exists 𝜆_𝑘𝑗 such that lim_𝑗→∞𝑔_𝑘𝑗+𝐴^𝑇_𝑘𝑗𝜆_𝑘𝑗=0. (76) The conclusion follows.

6. Some Numerical Experiments

(1)Updating of 𝐻_𝑘 is done by 𝐻_𝑘+1=𝐻_𝑘+(𝑦^𝑇_𝑘𝑦_𝑘/𝑦^𝑇_𝑘𝑠_𝑘)−(𝐻_𝑘𝑠_𝑘𝑠^𝑇_𝑘𝐻_𝑘/𝑠^𝑇_𝑘𝐻_𝑘𝑠_𝑘), where 𝑦_𝑘=𝜃_𝑘𝑦_𝑘+(1−𝜃_𝑘)𝐻_𝑘𝑠_𝑘𝜃_𝑘=1𝑠^𝑇_𝑘𝑦_𝑘≥0.2𝑠^𝑇_𝑘𝐻_𝑘𝑠_𝑘0.8𝑠^𝑇_𝑘𝐻_𝑘𝑠_𝑘𝑠^𝑇_𝑘𝐻_𝑘𝑠_𝑘−𝑠^𝑇_𝑘𝑦_𝑘 (77) and 𝑦_𝑘=𝑔_𝑘+1−𝑔_𝑘+(𝐴_𝑘+1−𝐴_𝑘)𝜌_𝑘, 𝑠_𝑘=𝑥_𝑘+1−𝑥_𝑘𝜌_𝑘 is the multipluser of corresponding quadratic subproblems.(2)We assume the error toleration is 10⁻⁵.(3)The algorithm parameters were set as follows: 𝐻₀=𝐼∈𝑅^𝑛×𝑛, 𝜂₁=0.25, 𝜂₂=0.25, 𝜂₃=0.1, 𝛼₁=𝛼₂=0.5, 𝛾=0.02, 𝜌=0.5, 𝜉=10⁻⁶, 𝛾₀=0.1, 𝛾₁=0.5, 𝛾₂=2, 𝛾_𝐹=10⁻⁴, Δ₀=1. The program is written in Matlab.

The numerical results for the test problems are listed in Table 1.

Table 1 

In Table 1, the problems are numbered in the same way as in Schittkowski [20] and Hock and Schittkowski [21]. For example, “S216” is the problem (216) in Schittkowski [20] and “HS6” is the problem (6) in Hock and Schittkowski [21]. NF, NG represent the numbers of function and gradient calculations and “L's” is the solution in [22]. The numerical results show that the our algorithm is more effective than the L's for most test examples. Moreover, the higher the level of nonmonotonic, the better the numerical results. The results show that the new algorithm is robust and effective, and more flexible for the acceptance of the the trial iterate.

Acknowledgments

This research is supported by the National Natural Science Foundation of China (no. 11101115) and the Natural Science Foundation of Hebei Province (nos. A2010000191, Y2012021).