A Filter Algorithm with Inexact Line Search

Abstract

A filter algorithm with inexact line search is proposed for solving nonlinear programming problems. The filter is constructed by employing the norm of the gradient of the Lagrangian function to the infeasibility measure. Transition to superlinear local convergence is showed for the proposed filter algorithm without second-order correction. Under mild conditions, the global convergence can also be derived. Numerical experiments show the efficiency of the algorithm.

1. Introduction

Fletcher and Leyffer [1] proposed filter methods in 2002 offering an alternative to traditional merit functions in solving nonlinear programming problems (NLPs). The underlying concept is that a trial point is accepted provided that there is a sufficient decrease of the objective function or the constraint violation. Filter methods avoid the difficulty of determining a suitable value of the penalty parameter in the merit function. The promising numerical results in [1] led to a growing interest in filter methods in recent years. Two variants of trust-region filter sequential quadratic programming (SQP) method were proposed by Fletcher et al. [2, 3]. Chin and Fletcher [4] developed filter method to sequential linear programming strategy that takes equality-constrained quadratic programming steps. Ribeiro et al. [5] proposed a general filter algorithm that does not depend on the particular method used for the step of computation. Ulbrich [6] argued superlinear local convergence of a filter SQP method. Ulbrich et al. [7] and Wächter and Biegler [8] applied filter technique to interior method and achieved the global convergence to first-order critical point. Wächter and Biegler [9, 10] proposed a line-search filter method and applied it to different algorithm framework. Gould et al. [11] and Shen et al. [12] developed new multidimensional filter technique. Su and Pu [13] extended the monotonicity of the filter technique. Nie [14] applied filter method to solve nonlinear complementarity problems. In this paper, the global convergence is analyzed widely. However, it has been noted by Fletcher and Leyffer [1] that the filter approach can suffer from the Maratos effect as that of a penalty function approach. By the Maratos effect, a full step can lead to an increase of both infeasibility measure and objective function in filter components even if arbitrarily close to a regular minimizer. This makes the full step unacceptable for the filter and can prohibit fast local convergence.

In this paper, we propose a filter algorithm with inexact line-search for nonlinear programming problems that ensures superlinear local convergence without second-order correction steps. We use the norm of the gradient of the Lagrangian function in the infeasibility measure in the filter components. Moreover, the new filter algorithm has the same global convergence properties as that of the previous works [2, 3, 9]. In addition, since the sufficient decrease conditions in an SQP framework can usually make the algorithm complex and time-consuming, the presented method is a line-search method without using SQP steps. An inexact line-search criterion is used as the sufficient reduction conditions. In the end, numerical experiences also show the efficiency of the new filter algorithm.

This paper is organized as follows. For the main part of the paper, the presented techniques will be applied to general NLP. In Section 2, we state the algorithm mechanism. The convergent properties are shown in Section 3. The global and superlinear convergence are proved. Furthermore, the Maratos effect is avoided. Finally, Section 4 shows the effectiveness of our method under some numerical experiences.

2. Inexact Line-Search Filter Approach

2.1. The Algorithm Mechanism

We describe and analyze the line-search filter method for NLP with equality constraints. State it asmin_{𝑥∈ℝ𝑛}𝑓(𝑥)subjectto𝑐_𝑖(𝑥)=0,𝑖∈𝐸, (2.1) where the objective function 𝑓∶ℝ^𝑛→ℝ and the constraints 𝑐_𝑖 are assumed to be continuously differentiable, and 𝐸={𝑖∣𝑖=1,2,…,𝑚}.

The corresponding Lagrangian function is𝐿(𝑥,𝜆)=𝑓(𝑥)+𝜆^𝑇𝑐(𝑥), (2.2) where the vector 𝜆 corresponds to the Lagrange multiplier. The Karush-Kuhn-Tucker (KKT) conditions for (2.1) are∇𝑓(𝑥)+𝜆^𝑇(𝑥)𝑐(𝑥)∇𝑐=0,=0. (2.3)

For a given initial estimate 𝑥₀, the line-search algorithm generates a sequence of iterates 𝑥_𝑘 by 𝑥_𝑘+1=𝑥_𝑘+𝛼_𝑘𝑑_𝑘 as the estimates of the solution for (2.1). Here, the search direction 𝑑_𝑘 is computed from the linearization at 𝑥_𝑘 of the KKT conditions (2.3):𝑊_𝑘∇𝑐𝑥_𝑘∇𝑐𝑥_𝑘0𝑑_𝑘𝜆⁺𝑥=−∇𝑓_𝑘𝑐𝑥_𝑘, (2.4) where the symmetric matrix 𝑊_𝑘 denotes the Hessian ∇²_𝑥𝑥𝐿(𝑥_𝑘,𝜆_𝑘) of (2.2) or a positive definite approximation to it.

After a search direction 𝑑_𝑘 has been computed, the step size 𝛼_𝑘∈(0,1] is determined by a backtracking line-search procedure, where a decreasing sequence of step size 𝛼_𝑘 is tried until some acceptable criteria are satisfied. Generally, the acceptable criteria are constructed by a condition that if the current trial point 𝑥_𝑘 can provide sufficient reduction of a merit function. The filter method proposed by Fletcher and Leyffer [1] offers an alternative to merit functions. In this paper, the filter notion is defined as follows.

Definition 2.1. A pair (𝑉_𝑘,f_𝑘) is said to dominate another pair (𝑉_𝑙,𝑓_𝑙) if and only if both 𝑉_𝑘≤𝑉_𝑙 and 𝑓_𝑘≤𝑓_𝑙.

Here, we define 𝑉(𝑥)∶=‖𝑐(𝑥)‖₂+‖∇𝐿(𝑥)‖₂ as the 𝑙₂ norm of the infeasibility measure. That is, we modify the infeasibility measure in filter, with this modification, the superlinear convergence is possibly derived. Strictly, it is a stationarity measure. However, we still call it infeasibility measure according to its function. In the rest of paper, the norm is always computed by 𝑙₂ norm excepting special noting.

Definition 2.2. A filter is a list of pairs (𝑉_𝑙,𝑓_𝑙) such that no pair dominates any other. A point (𝑉_𝑘,𝑓_𝑘) is said to be acceptable for inclusion in the filter if it is not dominated by any point in the filter.

When a pair (𝑉_𝑘,𝑓_𝑘) is said to be acceptable to the filter, we also say the iterate 𝑥_𝑘 is acceptable to the filter. In filter notion, a trial point 𝑥_𝑘+1 is accepted if it improves feasibility or improves the objective function. So, it is noted that filter criteria is less demanding than traditional penalty function. When improving optimality, the norm of the gradient of the Lagrangian function will tend to zero, so it offers a more precise analysis for the objective function.

However, this simple filter concept is not sufficient to guarantee global convergence. Fletcher et al. [3] replace this condition by requiring that the next iterate provides at least as much progress in one of the measure 𝑉 or 𝑓 that corresponds to a small fraction of the current infeasibility measure. Here, we use the similar technique to our filter. Formally, we say that a trial point can be accepted to the current iterate 𝑥_𝑘 or the filter if𝑉𝑥_𝑘≤𝛽𝑉𝑥_𝑙 (2.5a) or𝑓𝑥_𝑘≤𝑓𝑥_l−𝛾𝑉𝑥_𝑘, (2.5b)for some fixed constants 𝛽,𝛾∈(0,1), and (𝑉(𝑥_𝑙),𝑓(𝑥_𝑙)) are points in current filter. In practical implementation, the constants 𝛽 close to 1 and 𝛾 close to 0. However, the criteria (2.5a) and (2.5b) may make a trial point always provides sufficient reduction of the infeasibility measure alone, and not the objective function. To prevent this, we apply a technique proposed in Wächter and Biegler [10] to a different sufficient reduction criteria. The switching condition is∇𝑓^𝑇_𝑘𝑑_𝑘<0,−𝛼_𝑘∇𝑓^𝑇_𝑘𝑑_𝑘^𝑒1>𝛿𝑉𝑥_𝑘^𝑒2, (2.6) where 𝛿>0, 𝑒₂>1, 𝑒₁≥2𝑒₂. If the condition (2.6) holds, we replace the filter condition (2.5b) as an inexact line-search condition, that is, the Armijo type condition𝑓𝑥_𝑘+1≤𝑓𝑥_𝑘+𝜂𝛼_𝑘∇𝑓^𝑇_k𝑑_𝑘, (2.7) where 𝜂∈(0,1/2) is a constant. If (2.6) holds but not (2.7), the trial points are still determined by (2.5a) and (2.5b).

If a trial point 𝑥_𝑘 can be accepted at a step size by (2.7), we refer to 𝑥_𝑘 as an 𝑓 type iterate and the corresponding 𝛼_𝑘 as an 𝑓 step size.

2.2. The Algorithm Analysis

By the right part of switching condition (2.6), it ensures that the improvement to the objective function by the Armijo condition (2.7) is sufficiently large compared to the current infeasibility measure 𝑉(𝑥_𝑘). Thus, if iterate points remote from the feasible region, the decrease of the objective function can be sufficient. By setting 𝑒₂>1, the progress predicted by the line model −𝛼_𝑘[∇𝑓^𝑇_𝑘𝑑_𝑘]^𝑒1 of 𝑓 can be a power of the infeasibility measure 𝑉(𝑥). The choice of 𝑒₁≥2𝑒₂ makes it possible that a full step can be accepted by (2.7) when it closes to a local solution.

In this paper, we denote the filter as a set ℱ_𝑘 containing all iterates accepted by (2.5a) and (2.5b). During the optimization, if the 𝑓 type switching condition (2.6) holds and the Armijo condition (2.7) is satisfied, the trial point is determined by (2.7) not by (2.5a) and (2.5b), and the value of the objective function is strictly decreased. This can prevents cycling of the algorithm (see [10]).

If the linear system (2.4) is incompatible, no search direction 𝑑_𝑘 can be found and the algorithm switches to a feasibility restoration phase. In it, we try to decrease the infeasibility measure to find a new iterate 𝑥_𝑘+1 that satisfies (2.4) and is acceptable to the current filter. Similarly, in case ∇𝑓^𝑇_𝑘𝑑_𝑘<0, the sufficient decrease criteria for the objective function in (2.5b) may not be satisfied. To inspect where no admissible step size 𝛼 can be found and the feasibility restoration phase has to be invoked, we can consider reducing 𝛼 and define𝛼_𝑘^min=𝑥min1−𝛽,−𝛾𝑉_𝑘−∇𝑓^𝑇_𝑘𝑑_𝑘,𝛿𝑉𝑥_𝑘^𝑒2−∇𝑓^𝑇_𝑘𝑑_𝑘^𝑒1,if∇𝑓^𝑇_𝑘𝑑_𝑘<0,1−𝛽,otherwise. (2.8) If the trial step size 𝛼<𝛼_𝑘^min, the algorithm turns to the feasibility restoration phase.

By 𝛼_𝑘^min, it is ensured that the algorithm does not switch to the feasibility restoration phase as long as (2.6) holds for a step size 𝛼<𝛼_𝑘 and that the backtracking line-search procedure is finite. Thus, for a trial point 𝑥_𝑘, the algorithm eventually either delivers a new iterate 𝑥_𝑘+1 or reverts to the feasibility restoration phase. Once finding a feasible direction, the algorithm still implements the normal algorithm.

Of course, the feasibility restoration phase may not always be possible to find a point satisfying the filter-accepted criteria and the compatible condition. It may converge to a nonzero local minimizer of the infeasibility measure and indicate that the algorithm fails. In this paper, we do not specify the particular procedure for the feasibility restoration phase. Any method for dealing with a nonlinear algebraic system can be used to implement a feasibility restoration phase.

2.3. The Algorithm

We are now in the place to state the overall algorithm.

Algorithm 2.3. Step 1. Given: starting point 𝑥₀, constants 𝑉^max=max{10⁴,1.2𝑉(𝑥₀)}, 𝛽,𝛾∈(0,1), 𝜂∈(0,1/2), 𝛿>0, 𝑒₁≥2𝑒₂, 𝑒₂>1, 𝜏∈(0,1).Step 2. Initialize: ℱ₀={(𝑉^max,𝑓(𝑥₀))}, the iteration counter 𝑘←0.Step 3. For 𝑘=0,1,2,…, stop if 𝑥_𝑘 satisfies the KKT conditions (2.3).Step 4. Compute the search direction 𝑑_𝑘 from (2.4). If the system (2.4) is incompatible, go to the feasibility restoration phase in Step 7.Step 5. Set 𝛼₀=1, compute 𝛼_𝑘^min.(1)If 𝛼_𝑘<𝛼_𝑘^min, go to Step 7. Otherwise, compute the new trial point 𝑥_𝑘+1=𝑥_𝑘+𝛼_𝑘𝑑_𝑘.(2)If the conditions (2.6) and (2.7) hold, accept the trial step and go to Step 6, otherwise set 𝑥_𝑘=𝑥_𝑘+1, go to Step 5(3).(3)In case where no 𝛼_𝑘 make (2.7) hold, if 𝑥_𝑘+1 can be accepted to the filter, augment the filter by ℱ_𝑘+1=ℱ_𝑘∪{(𝑉(𝑥_𝑘+1),𝑓(𝑥_𝑘+1))}, go to Step 6; Otherwise set 𝑥_𝑘=𝑥_𝑘+1, go to Step 5(4).(4)Compute 𝛼_𝑘+1=𝜏𝛼_𝑘, go back to Step 5(1).Step 6. Increase the iteration counter 𝑘←𝑘+1 and go back to Step 4.Step 7. Feasibility restoration phase: by decreasing the infeasibility of 𝑉 to find a new iterate 𝑥_𝑘+1 such that (2.4) is compatible. And if (2.7) holds at 𝑥_𝑘+1, continue with the normal algorithm in Step 6; if (2.5a) and (2.5b) hold at 𝑥_𝑘+1, augment the filter by ℱ_𝑘+1=ℱ_𝑘∪{(𝑉(𝑥_𝑘+1),𝑓(𝑥_𝑘+1))}, and then continue with the normal algorithm in Step 6; if the feasibility restoration phase cannot find such a point, stop with insuccess.

Remark 2.4. In contrast to SQP method with trust-region technique, the actual step does not necessarily satisfy the linearization of the constraints.

Remark 2.5. Practical experience shows that the filter allows a large degree of nonmonotonicity and this can be advantageous to some problems.

Remark 2.6. To prevent the situation in which a sequence of points for which are 𝑓 type iterative point with 𝑉_𝑘→∞ is accepted, we set an upper bound 𝑉^max on the infeasibility measure function 𝑉.

For further specific implementation details of Algorithm 2.3, see Section 4.

3. Convergence Analysis

3.1. Global Convergence

In this section, we give a global convergence analysis of Algorithm 2.3. We refer to the global convergence analysis of Wächter and Biegler [10] in some places. First, state the necessary assumptions.

Assumption A1. Let all iterates 𝑥_𝑘 are in a nonempty closed and bounded set 𝑆 of ℝ^𝑛.

Assumption A2. The functions 𝑓 and 𝑐 are twice continuously differentiable on an open set containing 𝑆.

Assumption A3. The matrix 𝑊_𝑘 is positive definite on the null space of the Jacobian ∇𝑐(𝑥_𝑘) and uniformly bounded for all 𝑘, and the Lagrange multiply 𝜆 is bounded for all 𝑘.

Assumptions A1 and A2 are the standard assumptions. Assumption A3 plays an important role to obtain the convergence result and ensures that the algorithm is implementable.

For stating conveniently, we define a set 𝐽={𝑖∣𝑥_𝑖isacceptedtothefilter}. In addition, sometimes, it is need to revise 𝑊_𝑘 to keep it positive definite by some updating methods such as damped BFGS formula [15] or revised Broyden’s method [16].

From Assumptions A1–A3, we can get𝑑_𝑘≤𝑀_𝑑,𝑚_𝑊𝑑_𝑘²≤𝑑^𝑇_𝑘𝑊_𝑘𝑑_𝑘≤𝑀_𝑊𝑑_𝑘²,𝜆_𝑘≤𝑀_𝜆, (3.1) where 𝑀_𝑑, 𝑀_𝜆, 𝑀_𝑊, and 𝑚_𝑊 are constants.

Lemma 3.1. Suppose Assumptions A1–A3 hold, if {𝑥_𝑘𝑖} is a subsequence of iterates for which ‖𝑑_𝑘𝑖‖>𝜖₁ with a constant 𝜖₁>0 independent of 𝑘_𝑖, then for constant 𝜖₂=(𝑚_𝑊/2)𝜖₁, if 𝑉(𝑥_𝑘𝑖)≤(𝑚_𝑊/2𝑀_𝜆)𝜖₁, then ∇𝑓𝑥_𝑘𝑖^𝑇𝑑_𝑘𝑖≤−𝜖₂. (3.2)

Proof. By the linear system (2.4) and (3.1), ∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖=−𝜆^𝑇_𝑘𝑖∇𝑐𝑥_𝑘𝑖𝑑_𝑘𝑖−𝑑^𝑇_𝑘𝑖𝑊_𝑘𝑖𝑑_𝑘𝑖=𝜆^𝑇_𝑘𝑖𝑐𝑥_𝑘𝑖−𝑑^𝑇_𝑘𝑖𝑊_𝑘𝑖𝑑_𝑘𝑖≤𝑀_𝜆𝑉𝑥_𝑘𝑖−𝑚_𝑊𝑑_𝑘𝑖²≤𝑀_𝜆𝑚_𝑊2𝑀_𝜆𝜖₁−𝑚_𝑊𝜖₁=−𝜖₂. (3.3)

Lemma 3.1 shows that the search direction is a descent direction for the objective function when the trial points are sufficiently close to feasible region.

Lemma 3.2. Suppose Assumptions A1–A3 hold, and that there exists an infinite subsequence {𝑥_𝑘𝑖} of {𝑥_𝑘} such that conditions (2.6) and (2.7) hold. Then lim_𝑘→∞𝑉𝑥_𝑘𝑖=0. (3.4)

Proof. From Assumptions A1 and A2, we know that ∇𝑓 is bounded. Hence, it has with (3.1) that there exists a constant 𝑀_𝑚>0 such that ∇𝑓^𝑇_𝑘𝑑_𝑘≤𝑀_𝑚. (3.5) By (2.6) it has 𝛿𝑉𝑥_𝑘𝑖^𝑒2<−𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖^𝑒1≤𝑀^𝑒1_𝑚𝛼_𝑘𝑖. (3.6) As 1−1/𝑒₁>0, we have 𝛿𝑀^𝑒1_𝑚^1−1/𝑒1𝑉𝑥_𝑘𝑖^{𝑒2−𝑒2/𝑒1}<𝛼^1−1/𝑒1_𝑘𝑖. (3.7) Then by (2.7) and (3.7), 𝑓𝑥_𝑘𝑖+1−𝑓𝑥_𝑘𝑖≤𝜂𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖<−𝜂𝛿^1/𝑒1𝛼^1−1/𝑒1_𝑘𝑖𝑉𝑥_𝑘𝑖^𝑒2/𝑒1<−𝜂𝛿^1/𝑒1𝛿𝑀^𝑒1_𝑚^1−1/𝑒1𝑉𝑥_𝑘𝑖^𝑒2. (3.8) Hence, for 𝑐₁∶=𝜂𝛿^1/𝑒1(𝛿/𝑀^𝑒1_𝑚)^1−1/𝑒1, an integer 𝐾 and all 𝑗=1,2,…, 𝑓𝑥_𝐾+𝑗=𝑓𝑥_𝐾+^{𝐾+𝑗−1}_{𝑘𝑖=𝐾}𝑓𝑥_𝑘𝑖+1−𝑓𝑥_𝑘𝑖<𝑓𝑥_𝐾−𝑐₁^{𝐾+𝑗−1}_{𝑘𝑖=𝐾}𝑉𝑥_𝑘𝑖^𝑒2. (3.9) Since 𝑓(𝑥_𝐾+𝑗) is bounded below as 𝑗→∞, the series on the right hand side in the last line of (3.8) is bounded, then implies the conclusion.

Lemma 3.3. Let {𝑥_𝑘𝑖}⊂{𝑥_𝑘} be an infinite subsequence of iterates so that (𝑉(𝑥_𝑘𝑖),𝑓(𝑥_𝑘𝑖)) is entered into the filter. Then lim_𝑖→∞𝑉𝑥_𝑘𝑖=0. (3.10)

Proof. Here, we refer to the proof of [2, Lemma  3.3]. If the conclusion is not true, there exists an infinite subsequence {𝑘_𝑗}⊂{𝑘_𝑖}⊂𝐽 such that 𝑉𝑥_𝑘𝑗≥𝜖, (3.11) for all 𝑗 and for some 𝜖>0. This means that no other (𝑉,𝑓) pair can be added to the filter at a later stage within the region 𝑉_𝑘𝑗−1−𝛽𝜖,𝑉_𝑘𝑗×𝑓_𝑘𝑗−𝛾𝜖,𝑓_𝑘𝑗, (3.12) or with the intersection of this region with 𝑆₀=0,𝑉^max×𝑓^min,∞ (3.13) for some constants 𝑓^min≤𝑓(𝑥_𝑘). Now, the area of each of these regions is (1−𝛽)𝛾𝜖². Hence, the set 𝑆₀∪{(𝑉,𝑓)∣𝑓≤𝑀_𝑓} is completely covered by at most a finite number of such regions, for any 𝑀_𝑓≥𝑓^min. Since the pairs (𝑉_𝑘𝑗,𝑓_𝑘𝑗) keep on being added to the filter, 𝑓_𝑘𝑗 tends to infinity when 𝑖 tends to infinity. Without loss of generality, assume that 𝑓_𝑘𝑗+1≥𝑓_𝑘𝑗 for all 𝑗 is sufficiently large. But (2.5a) and (3.11) imply that 𝑉_𝑘𝑗+1≤𝛽𝑉_𝑘𝑗, (3.14) so 𝑉_𝑘𝑗→0, which contradicts (3.11). Then, this latter assumption is not true and the conclusion follows.

Next, we show that if {𝑥_𝑘} is bounded, there exists at least one limit point of the iterative points is a first-order optimal point for (2.1).

Lemma 3.4. Suppose Assumptions A1–A3 hold. Let {𝑥_𝑘𝑖} be a subsequence with ∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖<−𝜖₂ for a constant 𝜖₂>0 independent of 𝑘_𝑖. Then, there exists a constant 𝛼>0 so that for all 𝑘_𝑖 and 𝛼_𝑘𝑖<𝛼,𝑓𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖−𝑓𝑥_𝑘𝑖≤𝜂𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖. (3.15)

Proof. From Assumptions A1 and A2, 𝑑^𝑇∇²𝑓(𝑥)𝑑≤𝑐_𝑓‖𝑑‖² for some constant 𝑐_𝑓>0. Thus, it follows from the Taylor Theorem and (3.1) that𝑓𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖−𝑓𝑥_𝑘𝑖−𝛼_𝑘𝑖∇𝑓^𝑇_𝑘i𝑑_𝑘𝑖=𝛼²_𝑘𝑖𝑑^𝑇_𝑘𝑖∇²𝑓𝑦₁𝑑_𝑘𝑖≤𝑐_𝑓𝛼²_𝑘𝑖𝑑_𝑘𝑖²≤𝛼_𝑘𝑖1−𝜂𝜖₂≤−1−𝜂𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖, (3.16) if 𝛼_𝑘𝑖≤𝛼_𝑘𝑖∶=(1−𝜂)𝜖₂/𝑐_𝑓𝑀²_𝑑, where 𝑦₁ denotes some point on the line segment from 𝑥_𝑘𝑖 to 𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖. Then the conclusion follows.

Lemma 3.5. Suppose Assumptions A1–A3 hold, and the filter is augmented only a finite number of times, thenlim_𝑘→∞𝑑_𝑘=0. (3.17)

Proof. Since the filter is augmented only a finite number of times, there exists an integer 𝐾₁ so that for all iterates {𝑥_𝑘}_𝑘>𝐾1 the filter is not augmented. If the claim is not true, there must exist a subsequence {𝑥_𝑘𝑖} and a constant 𝜖>0 so that ‖𝑑_𝑘𝑖‖≥𝜖 for all 𝑖. Then by Lemma 3.1, it has ∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖≤−𝜖₂ for all 𝑘_𝑖≥𝐾₂, 𝐾₂ is some integer and 𝐾₂>𝐾₁. And from Lemmas 3.2 and 3.4, it has 𝑉(𝑥_𝑘𝑖)≤𝜖 and 𝑓𝑥_𝑘𝑖+1−𝑓𝑥_𝑘𝑖≤𝜂𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖≤−𝛼_𝑘𝑖𝜂𝜖₂. (3.18) Since 𝑓(𝑥_𝑘𝑖) is bounded below and monotonically decreasing for all 𝑘≥𝐾₂, one can conclude that lim_𝑖→∞𝛼_𝑘𝑖=0. This means that for 𝑘_𝑖>𝐾₂ the step size 𝛼=1 has not been accepted. So, we can get a 𝛼_𝑘𝑖<1 such that a trial point 𝑥_𝑘𝑖+1=𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖 satisfies 𝑉𝑥_𝑘𝑖+1,𝑓𝑥_𝑘i+1∉ℱ_𝑘𝑖 (3.19) or 𝑓𝑥_𝑘𝑖+1−𝑓𝑥_𝑘𝑖>𝜂𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖. (3.20) Let 𝑉^min=min{𝑉_𝑘∣(𝑉_𝑘,𝑓_𝑘)∈ℱ_𝑘}. From Lagrange’s Theorem, it has 𝑉𝑥_𝑘𝑖+1=𝑉𝑥_𝑘𝑖+𝛼_𝑘𝑖∇𝑉𝑦₂^𝑇𝑑_𝑘𝑖≤𝑉𝑥_𝑘𝑖+𝛼_𝑘𝑖∇𝑉𝑦₂𝑑_𝑘𝑖≤𝑉𝑥_𝑘𝑖+𝑐_𝑉𝛼_𝑘𝑖𝑀_𝑑, (3.21) for some constant 𝑐_𝑉, where 𝑦₂ denotes some point on the line segment from 𝑥_𝑘𝑖 to 𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖. Since lim_𝑖→∞𝛼_𝑘𝑖=0 and lim_𝑖→∞𝑉(𝑥_𝑘𝑖)=0 by Lemmas 3.2 and 3.3, it has 𝑉(𝑥_𝑘𝑖+1)<𝑉^min for 𝑘_𝑖 sufficiently large, so (3.19) is not true. In case (3.20), since 𝛼_𝑘𝑖→0 for sufficiently large 𝑘_𝑖, we have 𝛼_𝑘𝑖≤𝛼 with 𝛼 from Lemma 3.4, that is, (3.20) can not be satisfied. Then the conclusion follows.

Lemma 3.6. Suppose Assumptions  A1–A3 hold. Let {𝑥_𝑘𝑖} be a subsequence of {𝑥_𝑘} with ∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖≤−𝜖₂ for a constant 𝜖₂>0 independent of 𝑘_𝑖. Then, there exists trial points can be accepted to the filter.

Proof. The mechanisms of Algorithm 2.3 ensure that the first iterate can be accepted to the filter. Next, we can assume that (𝑉(𝑥_𝑘),𝑓(𝑥_𝑘)) is acceptable to the 𝑘th filter and (𝑉(𝑥_𝑙),𝑓(𝑥_𝑙))∈ℱ_𝑘,𝑙<𝑘. If 𝛼_𝑘𝑖≤𝑐₂∶=𝜖₂/𝑀²_𝑑𝑐_𝑓, it has 𝛼²_𝑘𝑖≤𝛼_𝑘𝑖𝜖₂𝑀²_𝑑𝑐_𝑓≤−𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖𝑐_𝑓𝑑_𝑘𝑖², (3.22) that is, 𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖+𝑐_𝑓𝛼²_𝑘𝑖‖𝑑_𝑘𝑖‖²≤0, so by (3.16) 𝑓𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖≤𝑓𝑥_𝑘𝑖. (3.23) Similarly, if 𝛼_𝑘𝑖≤𝑐₃𝑉(𝑥_𝑘𝑖)≤𝑉(𝑥_𝑘𝑖)/‖𝑑‖²𝑐_𝑉, with 𝑐₃∶=1/𝑀²_𝑑𝑐_𝑉 and 𝑐_𝑉 from Lemma 3.5, it has −𝛼_𝑘𝑖𝑉𝑥_𝑘𝑖+𝑐_𝑉𝛼²_𝑘𝑖𝑑_𝑘𝑖²𝑉𝑥≤0,_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖≤𝑉𝑥_𝑘𝑖. (3.24) Hence, we have 𝑓𝑥_𝑘𝑖+1=𝑓𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖≤𝑓𝑥_𝑘𝑖≤𝑓𝑥_𝑙−𝛾𝑉𝑥_𝑙,𝑉𝑥_𝑘𝑖+1=𝑉𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖≤𝑉𝑥_𝑘𝑖≤𝛽𝑉𝑥_𝑙. (3.25) The claim then follows from (3.25).

The last Lemma 3.6 shows, for case 𝑉(𝑥_𝑘)>0, Algorithm 2.3 either accepts a new iterate to the filter or switches to the feasibility restoration phase. For case 𝑉(𝑥_𝑘)=0 and the algorithm does not stop at a KKT point, then ∇𝑓^𝑇_𝑘𝑑_𝑘<0, 𝛼_𝑘^min=0, and the Armijo condition (2.7) is satisfied for sufficiently small step size 𝛼_𝑘, so an 𝑓 type iterate is accepted. Hence, the inner loop in Step 5 always terminates in a finite number of trial steps, and Algorithm 2.3 is well defined.

Lemma 3.7. Suppose Assumptions A1–A3 hold. Let {𝑥_𝑘𝑖} be a subsequence with ‖𝑑_𝑘𝑖‖≥𝜖 for a constant 𝜖>0 independent of 𝑘_𝑖. Then, there exists an sufficient large integer 𝐾 such that for all 𝑘_𝑖>𝐾 the algorithm can generate some trial points either be accepted to the filter or be 𝑓 type steps.

Proof. By Lemmas 3.1, 3.2, and 3.3, there exist constants 𝜖₁,𝜖₂>0 so that 𝑉𝑥_𝑘𝑖≤𝜖₁,∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖<−𝜖₂ (3.26) for all 𝑘_𝑖>𝐾.
If 𝑉(𝑥_𝑘𝑖)=0, the 𝑓 type switching condition (2.6) is true, there must exist iterates for which are 𝑓 type iterates. For the remaining iterates with 𝑉(𝑥_𝑘𝑖)>0, if𝑉𝑥_𝑘𝑖<min𝛼𝑐₃,𝑐₂𝑐₃,𝜏𝑐₃𝜖^𝑒1₂𝛿^1/𝑒2−1, (3.27) with 𝛼 from Lemma 3.4 and 𝑐₂,𝑐₃ from Lemma 3.6, it implies with 𝑒₂>1𝛿𝑉𝑥_𝑘𝑖^𝑒2𝜖^𝑒1₂<𝜏𝑐₃𝑉𝑥_𝑘𝑖, (3.28) as well as 𝑐₃𝑉𝑥_𝑘𝑖<min𝛼,𝑐₂. (3.29) Now choose an arbitrary 𝑘_𝑖≥𝐾 with 𝑉(𝑥_𝑘𝑖)>0 and define 𝑐₄∶=𝑐₃𝑉𝑥_𝑘𝑖=min𝛼,𝑐₂,𝑐₃𝑉𝑥_𝑘𝑖. (3.30) Lemmas 3.4 and 3.6 then imply that a trial step size 𝛼_𝑘𝑖≤𝑐₄ satisfies both 𝑓𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖≤𝑓𝑥_𝑘𝑖+𝜂𝛼_𝑘𝑖∇𝑓^𝑇_𝑘𝑖𝑑_𝑘𝑖,𝑉𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖,𝑓𝑥_𝑘𝑖+𝛼_𝑘𝑖𝑑_𝑘𝑖∈ℱ_𝑘𝑖. (3.31)
Since 𝛼>𝜏𝛼_𝑘>𝜏𝛼_𝑘^min by the definition of 𝛼_𝑘^min, the method does not switch to the feasibility restoration phase for those trial step sizes. Then the claim follows.

Based on the above lemmas, we can give the main global convergence result.

Theorem 3.8. Suppose Assumptions A1–A3 hold, then lim_𝑘→∞𝑉𝑥_𝑘=0,lim_𝑘→∞𝑑_𝑘=0, (3.32)(3.33) that is, there exits a limit point 𝑥 of {𝑥_𝑘} which is a first-order optimal point for (2.1).

Proof. (3.32) follows from Lemmas 3.2 and 3.3.
If the filter is augmented a finite number of times, then the claim (3.33) holds from Lemma 3.5. For either case, there exists a subsequence {𝑥_𝑘𝑖} so that 𝑘_𝑖∈𝐽 for all 𝑖. Suppose the conclusion (3.33) is not true, there must exist a subsequence {𝑥_𝑘𝑗} of {𝑥_𝑘𝑖} such that ‖𝑑_𝑘𝑗‖≥𝜖 for some constant 𝜖>0. Hence by Lemmas 3.1 and 3.3, it has ∇𝑓^𝑇_𝑘𝑗𝑑_𝑘𝑗<−𝜖₂ and lim_𝑖→∞𝑉(𝑥_𝑘𝑗)=0 for all 𝑘_𝑗. Then by Lemma 3.7, when 𝛼<min{𝛼,𝑐₂,𝑐₃𝑉(𝑥_𝑘𝑗)}, the algorithm can generate a 𝑓 type iterate, that is, the filter is not augmented, this contradicts the choice of {𝑥_𝑘𝑗}, so that (3.33) holds.

3.2. Local Convergence

In this section, we show the local convergence of Algorithm 2.3.

Assumption A4. The iterates 𝑥_𝑘 converge to a point 𝑥 that satisfies𝑉(𝑥)=0,∇_𝑥𝐿𝑥,𝜆=0,∇_𝑥𝑥𝐿𝑥,𝜆ispositivedefiniteon𝑑∶∇𝑐(𝑥)^𝑇,(𝑑=0∇𝑐𝑥)hasfull-rowrank. (3.34)

Assumption A5. There is a neighborhood 𝑁(𝑥) of 𝑥 such that 𝑊_𝑘=∇_𝑥𝑥𝐿(𝑥_𝑘,𝜆_𝑘), for all 𝑥_𝑘∈𝑁(𝑥).

Remark 3.9. Under Assumption A4, the point 𝑥 is a strict local minimum of (2.1).

Remark 3.10. Under Assumptions A4 and A5, it is well known that with the choice 𝑥_𝑘+1=𝑥_𝑘+𝑑_𝑘, the sequence {𝑥_𝑘} converges 𝑞-superlinearly to 𝑥 and that the convergence is 𝑞-quadratic if ∇_𝑥𝑥𝑓 and ∇_𝑥𝑥𝑐_𝑖 are lipschitz continuous in a neighborhood of 𝑥. That is, for any given 𝜁∈[0,1], 𝑥_𝑗∈𝑁(𝑥), 𝑗=𝑘,𝑘+1,…, and 𝑥_𝑗+1=𝑥_𝑗+𝑑_𝑗, it has 𝑑_𝑗+1≤𝜁𝑑_𝑗,𝑥_𝑗+1−𝑥≤𝜁𝑥_𝑗−𝑥. (3.35)
We use the proof techniques of local convergence in [6]. In proof, define 𝑙_𝜌(𝑥,𝜆)=𝐿(𝑥,𝜆)+(𝜌/2)‖𝑐(𝑥)‖²₂ and 𝑙_𝜌̂(𝑥,𝜆)=𝑓(𝑥)+(𝜌/2)𝑉(𝑥) with 𝜌 is a parameter.

Lemma 3.11. Suppose Assumptions A1–A3 hold. Let 𝑥 satisfy the Assumption A4. Then, there exist constants 0<𝑟<𝑡, 𝜌₀>0 and a neighborhood 𝑁_𝜎(𝑥)={𝑥∶‖𝑥−𝑥‖<𝜎}⊂𝑁(𝑥) such that 𝑟2‖𝑥−𝑥‖²+𝜌−𝜌₀2‖𝑐(𝑥)‖²≤𝑙_𝜌(𝑥,𝜆)−𝑙_𝜌𝑥,𝜆≤𝑡1+𝜌‖𝑥−𝑥‖², (3.36) for all 𝜌≥𝜌₀ and all 𝑥∈𝑁_𝜎(𝑥).
Moreover, possibly after increasing 𝑡>0,𝜌₀>0 and reducing 𝜎>0, we have that for all 𝜌≥𝜌₀ and all 𝑥∈𝑁_𝜎(𝑥)𝑟2‖𝑥−𝑥‖²+𝜌−𝜌₀2𝑉(𝑥)≤𝑙_𝜌̂(𝑥,𝜆)−𝑙_𝜌̂𝑥,𝜆≤𝑡1+𝜌‖𝑥−𝑥‖². (3.37)

Proof. Let 𝑥∈𝑁(𝑥). Using Taylor’s Theorem and ∇_𝑥𝑙_𝜌(𝑥,𝜆)=0,∇_𝜆𝑙_𝜌(𝑥,𝜆)=0, we have with some (𝑥′,𝜆′) on the line segment between (𝑥,𝜆) and (𝑥,𝜆)𝑙_𝜌(𝑥,𝜆)−𝑙_𝜌𝑥,𝜆=12(𝑥−𝑥)^𝑇∇_𝑥𝑥𝑙_𝜌𝑥^′,𝜆^′(𝑥−𝑥)+(𝑥−𝑥)^𝑇∇_𝑥𝜆𝑙_𝜌𝑥^′,𝜆^′𝜆−𝜆. (3.38) Obviously, it has ∇_𝑥𝑥𝑙_𝜌𝑥^′,𝜆^′=∇_𝑥𝑥𝑙_𝜌/2𝑥^′,𝜆^′+𝜌2𝑐𝑥^′+𝜌2∇𝑐𝑥^′𝑇∇𝑐𝑥^′. (3.39) Under Assumption A4, there exists 𝜌>0 such that for all 𝜌≥𝜌, 𝑑^𝑇∇_𝑥𝑥𝑙_𝜌/2𝑥,𝜆𝑑≥4𝑟‖𝑑‖²,∀𝑑∈ℝ^𝑛, (3.40) with a constant 𝑟>0, see [15, Theorem  17.5].
Suppose 𝜆(𝑥) is Lipschitz continuous and 𝐿_𝑦 is the Lipschitz constant of 𝜆, and 𝜌>0 is a constant. Let 𝜌₀∶=max{𝜌,4𝐿²_𝑦/𝑟} for all 𝑥 with ‖𝑥−𝑥‖≤𝜎, it has by continuity𝑑^𝑇∇_𝑥𝑥𝑙_𝜌0/2𝜌𝑥,𝜆+⁰2𝑐(𝑥)𝑑≥2𝑟‖𝑑‖²,∀𝑑∈ℝ^𝑛. (3.41) Thus, we obtain for all 𝑥∈𝑁_𝜎(𝑥) by (3.38), (3.39), and (3.41) 𝑙_𝜌0(𝑥,𝜆)−𝑙_𝜌0𝑥,𝜆≥𝑟‖𝑥−𝑥‖²+𝜌₀4∇𝑐𝑥^′(𝑥−𝑥)²+𝜆−𝜆^𝑇∇𝑐𝑥^′(𝑥−𝑥). (3.42) It is obvious for all 𝑠>0 that 2𝜆−𝜆^𝑇∇𝑐𝑥^′(𝑥−𝑥)1≥−𝑠∇𝑐𝑥^′(𝑥−𝑥)²−𝑠𝜆−𝜆²1≥−𝑠∇𝑐𝑥^′(𝑥−𝑥)²−𝑠𝐿²_𝑦𝜆−𝜆². (3.43) If 𝑠=𝑟/𝐿²_𝑦, 1/𝑠=𝐿²_𝑦/𝑟≤𝜌⁰/4, then it has 𝑙_𝜌0(𝑥,𝜆)−𝑙_𝜌0𝑥,𝜆≥𝑟2‖𝑥−𝑥‖²+𝜌₀8∇𝑐𝑥^′(𝑥−𝑥)². (3.44) Since 𝑐(𝑥)=0, 𝑙_𝜌(𝑥,𝜆)=𝑙_𝜌0(𝑥,𝜆)=𝑓(𝑥), and by (3.44) it has 𝑙_𝜌(𝑥,𝜆)−𝑙_𝜌𝑥,𝜆=𝑙_𝜌0(𝑥,𝜆)−𝑙_𝜌0𝑥,𝜆+𝜌−𝜌₀2‖𝑐(𝑥)‖²≥𝑟2‖𝑥−𝑥‖²+𝜌−𝜌₀2‖𝑐(𝑥)‖², (3.45) that is the left inequality in (3.36). For the right inequality in (3.36), it is obvious from (3.38) that for all 𝑥∈𝑁(𝑥), 𝑙_𝜌(𝑥,𝜆)−𝑙_𝜌𝑥,𝜆≤𝑡1+𝜌‖𝑥−𝑥‖², (3.46) with a constant 𝑡>0. This proves the inequality (3.36).
Choose 𝐾 large enough such that for 𝑗≥𝑘≥𝐾, 𝑐(𝑥_𝑗)→0. We can assume ‖𝑐(𝑥)‖<1. Then, if 𝜌₀ is large enough, it has from (3.44) and 𝑙_𝜌̂(𝑥,𝜆)=𝑙_𝜌0(𝑥,𝜆)=𝑓(𝑥) that𝑙̂_𝜌0(𝑥,𝜆)−𝑙̂_𝜌0𝑥,𝜆≥𝑙_𝜌0(𝑥,𝜆)−𝑙_𝜌0𝑥,𝜆≥𝑟2‖𝑥−𝑥‖². (3.47) By an analogue of (3.45) holds for 𝑙_𝜌̂, this proves the left inequality in (3.37).
On the other hand, it has𝑙_𝜌̂(𝑥,𝜆)−𝑙_𝜌̂𝑥,𝜆=𝑓(𝑥)+𝜌2𝑉(𝑥)−𝑓(𝑥). (3.48) Since 𝑓(𝑥) and 𝑐(𝑥) are twice continuously differentiable on closed set 𝑆, we have 𝑓(𝑥)−𝑓(𝑥)=𝑂(‖𝑥−𝑥‖²) and 𝑉(𝑥)=𝑂(‖𝑥−𝑥‖²). This shows the right inequality in (3.37) possibly after increasing 𝑡.

Lemma 3.12. Let 𝑥 satisfy Assumptions A4 and A5. Then for any 𝜁∈[0,1] and 𝑀≥1, there is an index 𝐾 such that for all 𝑘≥𝐾, with 𝑥_𝑘−𝑥≤𝑀min_{𝑉𝑙,𝑓𝑙∈ℱ𝑘}𝑥_𝑙−𝑥, (3.49) the points 𝑥_𝑗+1, 𝑗=𝑘,𝑘+1,…, with 𝛼_𝑘=1 are acceptable to ℱ_𝑗∶=ℱ_𝑘∪𝑉_𝑘,f_𝑘∪𝑉_𝑘+1,𝑓_𝑘+1∪⋯∪𝑉_𝑗,𝑓_𝑗. (3.50)

Proof. Let 𝑁(𝑥) as in Assumption A5,𝜁=1/2,𝜌₀ and 𝑁_𝜎(𝑥)⊂𝑁(𝑥) be given by Lemma 3.11. For all 𝑘≥𝐾₁, 𝐾₁ is a sufficient large integer, and choose 𝜌≥𝜌₀ so large that 𝛽𝜌2𝜌−𝛾≥𝛽4,𝜌−𝜌₀2≥𝛽1−4𝜌2. (3.51) Then, it has from (3.37) in Lemma 3.11𝑟2‖𝑥−𝑥‖²+𝛽1−4𝜌2𝑉(𝑥)=𝑙_𝜌̂(𝑥,𝜆)−𝑙_𝜌̂𝑥,𝜆≤𝑡1+𝜌‖𝑥−𝑥‖². (3.52) Let 𝑉^′∶=min_{𝑉𝑗,𝑓𝑗∈ℱ𝐾1}𝑉_𝑗. (3.53) By 𝑉^′>0 and continuity there exists 0<𝜎₁<𝜎 such that 𝑉(𝑥)≤𝛽𝑉′ for all 𝑥∈𝑁_𝜎1(𝑥), so the point 𝑥 is acceptable to ℱ_𝐾1. Since 𝑥_𝑘→𝑥, 𝑥_𝑘∈𝑁_𝜎1(𝑥) for all 𝑘≥𝐾₂>𝐾₁, 𝐾₂ is an integer. By (3.35), we can choose 𝜎₁ so small that for all 𝑘≥𝐾₂, the sequence {𝑥_𝑗}_𝑗≥𝑘 with 𝛼_𝑘=1 converges linearly with a contraction factor of at least 𝑥_𝑗+1−𝑥≤121−1−𝛽/2𝑟1−𝛽/42𝑀²𝑡𝑥1+𝜌^𝑗−𝑥≤12𝑥_𝑗−𝑥. (3.54) Suppose an arbitrary 𝑘₂≥𝐾₂ such that (3.49) holds, and set 𝜎₂∶=‖𝑥_𝑘−𝑥‖. By (3.54), it has 𝑥_𝑘+1∈𝑁_𝜎2(𝑥) and 𝑥_𝑘+1 is acceptable to ℱ_𝐾1. Next, it is a need to show that 𝑥_𝑘 is acceptable with respect to (𝑉_𝑙,𝑓_𝑙)∈ℱ_𝑘∪(𝑉_𝑘,𝑓_𝑘) for 𝐾₁≤𝑙≤𝑘. By (3.49), it has 𝑥_𝑙∈𝑁_𝜎(𝑥)⧵𝑁_𝜎2/𝑀(𝑥), so by (3.52) 𝑙_𝜌̂𝑥_𝑙,𝜆_𝑙−𝑙_𝜌̂𝑥,𝜆≥𝑟2𝑀²𝜎²₂+𝛽1−4𝜌2𝑉_𝑙. (3.55) And, by (3.52) and (3.54) 𝑙_𝜌̂𝑥_𝑘+1,𝜆_𝑘+1−𝑙_𝜌̂𝑥,𝜆≤𝑡1+𝜌𝑥_𝑘+1−𝑥²≤121−1−𝛽/2𝑟1−𝛽/42𝑀²𝜎²₂. (3.56) Next, suppose (𝑥,𝜆) with 𝑥∈𝑁_𝜎2(𝑥) is not acceptable to (𝑉_𝑙,𝑓_𝑙) then 𝑉(𝑥)>𝛽𝑉_𝑙,𝑓(𝑥)+𝛾𝑉(𝑥)>𝑓_𝑙. (3.57) Thus, it has with (3.51) and (3.52) 𝑙_𝜌̂(𝑥,𝜆)=𝑓(𝑥)+𝜌2𝑉(𝑥)>𝑓_𝑙+𝜌2−𝛾𝑉(𝑥)>𝑓_𝑙𝜌+𝛽2−𝛾𝑉_𝑙≥𝑓_𝑙𝜌+𝛽4𝑉_𝑙=𝑙_𝜌̂𝑥_𝑙,𝜆_𝑙−𝛽1−2𝜌2𝑉_𝑙≥𝑙_𝜌̂𝑥_𝑙,𝜆_𝑙−1−𝛽/2̂𝑙1−𝛽/4^𝜌𝑥_𝑙,𝜆_𝑙−𝑙_𝜌̂𝑥,𝜆. (3.58) This shows with (3.55) that 𝑙_𝜌̂(𝑥,𝜆)−𝑙_𝜌̂𝑥,𝜆>1−1−𝛽/2𝑙̂1−𝛽/4^𝜌𝑥_𝑙,𝜆_𝑙−𝑙_𝜌̂𝑥,𝜆≥1−1−𝛽/2𝑟1−𝛽/42𝑀²𝜎²₂. (3.59) This produces a contradiction to (3.56), so 𝑥_𝑘+1 is acceptable to ℱ_𝑘∪(𝑉_𝑘,𝑓_𝑘). Then, the acceptability of (𝑉_𝑗+1,𝑓_𝑗+1) for ℱ_𝑙 follows by induction.

Next, we show that the sequence {𝑥_𝑗}_𝑗≥𝑘 with 𝛼_𝑘=1 can make the sufficient decreasing condition (2.7) hold.

Lemma 3.13. Suppose Assumptions A1–A3 hold. Let 𝑥 satisfy Assumptions A4 and A5 and let 𝐾 be as in Lemma 3.11. Then for all 𝑘>𝐾 the sequence {𝑥_𝑗}_𝑗≥𝐾 with 𝛼_𝑗=1 satisfies 𝑓𝑥_𝑗+1≤𝑓𝑥_𝑗+𝜂𝛼_𝑗∇𝑓^𝑇_𝑗𝑑_𝑗. (3.60)

Proof. Suppose 𝛼_𝑗∇𝑓^𝑇_𝑗𝑑_𝑗<0 and 𝛼_𝑗[∇𝑓^𝑇_𝑗𝑑_𝑗]^𝑒1<−𝛿[𝑉(𝑥_𝑗)]^𝑒2 hold. By 𝛼_𝑗=1, thus ∇𝑓^𝑇_𝑗𝑑_𝑗<−(𝛿)^1/𝑒1𝑉𝑥_𝑗^𝑒2/𝑒1. (3.61)
On the other hand, with 𝛼_𝑗=1 the assertion 𝑓(𝑥_𝑗+1)≤𝑓(𝑥_𝑗)+𝜂∇𝑓^𝑇_𝑗𝑑_j yields𝜂∇𝑓^𝑇_𝑗𝑑_𝑗≥∇𝑓^𝑇_𝑗𝑑_𝑗+12𝑑^𝑇_𝑗∇²_𝑥𝑥𝑓𝑥^′𝑑_𝑗. (3.62) Thus, ∇𝑓^𝑇_𝑗𝑑_𝑗1≤−2𝑑1−𝜂^𝑇𝑗∇²_𝑥𝑥𝑓𝑥^′𝑑_𝑗, (3.63) where 𝑥′ on the line segment between 𝑥_𝑗 and 𝑥_𝑗+1.
Obviously, we can prove the conclusion if with 𝐾>0 large enough and for all 𝑗≥𝑘≥𝐾 the following holds−(𝛿)^1/𝑒1𝑉𝑥_𝑗^𝑒2/𝑒11≤−2𝑑1−𝜂^𝑇𝑗∇²_𝑥𝑥𝑓𝑥^′𝑑_𝑗. (3.64)
Since 𝑐_𝑗+∇𝑐^𝑇_𝑗𝑑_𝑗=0, 𝑑_𝑗=−∇𝑐^𝑇_𝑗(∇𝑐_𝑗∇𝑐^𝑇_𝑗)⁻¹𝑐_𝑗. By Assumption A4, ∇𝑐_𝑗 has full-row rank, there exists 𝑐_𝑑>0 such that𝑑_𝑗≤𝑐_𝑑𝑉𝑥_𝑗, (3.65) thus, 𝑑_𝑗²≤𝑐²_𝑑𝑉𝑥_𝑗². (3.66)
Choose 𝐾 large enough such that 𝑉(𝑥_𝑗)≤1 for all 𝑗≥𝑘≥𝐾. By 𝑒₁>2𝑒₂ and (3.66) it has𝑑_𝑗²≤𝑐²_𝑑𝑐₅𝑉𝑥_𝑗^𝑒2/𝑒1, (3.67) for a constant 𝑐₅∶=2𝛿^1/𝑒1(1−𝜂)/𝑐_𝑓𝑐²_𝑑, we can choose suitable parameters such that the last inequality holds. Thus, 𝛿^1/𝑒1𝑉𝑥_𝑗^𝑒2/𝑒1≥12𝑐1−𝜂^𝑓𝑑_𝑗²≥12𝑑1−𝜂^𝑇𝑗∇²_𝑥𝑥𝑓𝑥^′𝑑_𝑗. (3.68) This completes the proof.

Theorem 3.14. Suppose Assumptions A1–A5 hold. Then, there exists 𝐾>0 such that Algorithm 2.3 takes steps with 𝛼_𝑘=1 for all 𝑘≥𝐾, that is, 𝑥_𝑘+1=𝑥_𝑘+𝑑_𝑘. (3.69) In particular, 𝑥_𝑘 converges 𝑞-superlinearly to 𝑥. If ∇_𝑥𝑥𝑓 and ∇_𝑥𝑥𝑐_𝑖 are Lipschitz continuous in a neighborhood of 𝑥 then 𝑥_𝑘 converges 𝑞-quadratically.

Proof. Since Assumptions A4 and A5 hold, 𝑥_𝑘→𝑥 with 𝑥 satisfying A4. By Lemmas 3.12 and 3.13, the iterate (𝑥_𝑘+1,𝜆_𝑘+1)=(𝑥_𝑘+𝑑_𝑘,𝜆(𝑥_𝑘+𝑑_𝑘)) is acceptable to the filter ℱ_𝑘∪(𝑉_𝑘,𝑓_𝑘) and satisfies the sufficient decreasing condition (2.7). Thus, the trial iterate (𝑥_𝑘+𝑑_𝑘,𝜆(𝑥_𝑘+𝑑_𝑘)) is accepted by the algorithm and it has 𝑥_𝑘+1=𝑥_𝑘+𝑑_𝑘,𝜆_𝑘+1=𝜆𝑥_𝑘+𝑑_𝑘. (3.70) That is in both cases the algorithm takes the steps with 𝛼_𝑘=1. And according to Remark 3.10, {𝑥_𝑘} converges 𝑞-superlinearly to 𝑥.

4. Numerical Experience

In this section, we give some numerical results of Algorithm 2.3. We take some CUTE problems [17], which are available freely on NEOS, to test our algorithm. The test codes are edited in MATLAB. The details about the implementation are described as follows.(a)The parameters are set to 𝛽=0.99, 𝛾=0.0001, 𝜂=0.4, 𝛿=1, 𝜏=0.6, e₁=2.3, 𝑒₂=1.1, the termination tolerance 𝜖=1𝐸−6.(b)The optimal residual is defined as res=max∇𝑓𝑥_𝑘+∇𝑐𝑥_𝑘𝜆,𝑉𝑥_𝑘. (4.1) That is, the algorithm terminates when res<𝜖.(c)𝑊_𝑘 is updated by damped BFGS formula [15].

The detailed results of the numerical test on small-scale problems are summarized in Table 2. For comparison purposes, we also give the numerical results of tridimensional line-search filter solver (Tri-filter) in Shen et al. [12] in Table 2. The row headers in Tables 2 and 3 are presented in Table 1.

Table 1 

Description on headers.

Table 2 

Numerical results of small-scale problems.

Table 3 

Numerical results of mid-scale problems.

The results in Table 2 indicate that Algorithm 2.3 has a good effect.

In addition, we also test some mid-scale problems. And we compare the numerical results which are summarized in Table 3 in Algorithm 2.3 and SNOPT solver in Gill et al. [18] since no mid-scale problems are given in trifilter solver.

From Table 3, we find the efficiency of Algorithm 2.3 is also improved significantly. From both Tables 2 and 3, in general, the behavior of the proposed algorithm is rather stable. Finally, we may conclude that, as far as our limited computational experience is concerned, the proposed algorithm is well comparable to trifilter solver and SNOPT solver.

Acknowledgment

This research is partially supported by the National Natural Science Foundation of China (Grant nos. 10771162 and 91024021).