A Globally Convergent Line Search Filter SQP Method for Inequality Constrained Optimization

Abstract

A line search filter SQP method for inequality constrained optimization is presented. This method makes use of a backtracking line search procedure to generate step size and the efficiency of the filter technique to determine step acceptance. At each iteration, the subproblem is always consistent, and it only needs to solve one QP subproblem. Under some mild conditions, the global convergence property can be guaranteed. In the end, numerical experiments show that the method in this paper is effective.

1. Introduction

In this paper, we consider the following nonlinear inequality constrained optimization problem: min𝑓(𝑥)s.t.𝑔_𝑗(𝑥)≤0,𝑗∈𝐼={1,2,…,𝑚}, (P) where 𝑥∈𝑅^𝑛,𝑓∶𝑅^𝑛→𝑅, and 𝑔_𝑗(𝑗∈𝐼)∶𝑅^𝑛→𝑅¹ are assumed to be continuously differentiable.

It is well known that the sequential quadratic programming (SQP) method is one of the most efficient methods to solve problem (P). Because of its superlinear convergence rate, it has been widely studied. See Boggs and Tolle [1] for an excellent literature survey. In SQP methods, at each iteration the search direction is generally obtained by solving the subproblem as follows: min∇𝑓𝑥_𝑘^𝑇1𝑑+2𝑑^𝑇𝐵_𝑘𝑑s.t.𝑔_𝑗𝑥_𝑘+∇𝑔_𝑗𝑥_𝑘^𝑇𝑑≤0,𝑗∈𝐼={1,2,…,𝑚}, (1) where 𝐵_𝑘∈𝑅^𝑛×𝑛 is a symmetric positive definite matrix.

However, the previous QP subproblem has a serious shortcoming that constraints in (1) may be inconsistent. To overcome this disadvantage, much attention has been paid [2–7]. Burke and Han [2], Zhou [3], and J.-L. Zhang and X.-S. Zhang [4] modified the constraints in subproblem (1) to ensure that a revised QP subproblem is consistent, and their methods are globally convergent. Burke and Han's method is just a conceptual method and cannot be implementable practically, Zhou's method is based on the exact line search, and Zhang's method focuses on the inexact line search. Furthermore, Pantoja and Mayne [5] presented a modification of SQP algorithm by modifying both constraints and objective function in (1), and the search direction is obtained by solving a new QP subproblem which always has an optimal solution. In addition, Liu and Li [6] and Liu and Zeng [7] proposed SQP algorithms with cautious update criteria, which can be considered as modifications of the SQP algorithm given in [5].

In the previous methods [2–7], a penalty function is always used as a merit function to test the acceptability of the iterate points. However, as we all know, there are several difficulties associated with the use of penalty function and in particular with the choice of the penalty parameter. Too low a choice may result in the loss of an optimal solution; on the other hand, too large a choice damps out the effect of the objective function. Therefore, filter method was first introduced in trust region SQP method for constrained nonlinear optimization problems by Fletcher and Leyffer [8], offering an alternative to merit functions, as a tool to guarantee global convergence. Subsequently, global convergences of filter SQP methods were established by Fletcher et al. [9, 10]. Furthermore, Wächter and Biegler [11, 12] presented line search filter methods for nonlinear equality constrained programming, and the global convergence and local convergence were given. The promising numerical results of filter methods have led to a growing interest in filter methods in recent years [13–26], in which trust region strategy is used in [15, 26], and line search technique is taken in others. It is noteworthy that, based on [2–4] and line search filter technique, Su and Che [13] and Su [14] presented modified SQP filter methods for inequality constrained optimization. In Su's methods, the subproblem is always consistent, and the global convergence property is obtained.

In this paper, derived from [5–7] and the filter technique, especially the filter line search technique provided by Wächter and Biegler [11, 12], we propose a line search filter SQP method for inequality constrained optimization. This method has the following merits: it avoids using the penalty function to guarantee global convergence and has no requirement on initial point; at each iterate point the subproblem is always consistent, and it only needs to solve one QP subproblem; under some mild conditions, we prove that the algorithm either terminates at a Karush-Kuhn-Tucker (KKT) point within finite steps or generates an infinite sequence whose at least one cluster is a KKT point; in the end, numerical experiments show that our method is effective.

This paper is organized as follows. In the next section, the structure of line search filter technique is discussed. The algorithm is put forward in Section 3, and the global convergence theory of the algorithm is presented in Section 4. The numerical results for some typical examples are listed in Section 5. Finally in Section 6, a brief discussion on the proposed algorithm is given.

2. The Structure of Line Search Filter Technique

In this section, we will present the structure of line search filter technique and some related strategies. Instead of combing the objective function and constraint violation into a single function, filter methods view nonlinear optimization as a biobjective optimization problem that minimizes the objective function and the constraint violation. Now we give the structure of line search filter technique.

In the line search filter technique, after a search direction 𝑑_𝑘 has been computed, a step size 𝛼_𝑘,𝑙∈(0,1] is determined in order to obtain the trial iteration 𝑥_𝑘(𝛼_𝑘,𝑙)=𝑥_𝑘+𝛼_𝑘,𝑙𝑑_𝑘. More precisely, for fixed constants 𝛾_𝑓,𝛾_𝜃∈(0,1), a trial step size 𝛼_𝑘,𝑙 provides sufficient reduction with respect to the current point 𝑥_𝑘 if 𝜃𝑥_𝑘𝛼_𝑘,𝑙≤1−𝛾_𝜃𝜃𝑥_𝑘or𝑓𝑥_𝑘𝛼_𝑘,𝑙≤𝑓𝑥_𝑘−𝛾_𝑓𝜃𝑥_𝑘, (2) where 𝜃(𝑥) is a constraint violation function. Similar to [18], let 𝜃(𝑥_𝑘)=max_{1≤𝑖≤𝑚}[𝑔_𝑖(𝑥_𝑘)]₊ and [⋅]₊=max{⋅,0}.

For the sake of a simplified notation, the filter is defined in this paper not as a list but as a set ℱ_𝑘⊆[0,∞]×[−∞,∞] containing all (𝜃,𝑓) pairs that are prohibited in iteration 𝑘. We say that a trial point 𝑥_𝑘(𝛼_𝑘,𝑙) is acceptable to the filter if its (𝜃,𝑓) pair does not lie in the taboo region, that is, if 𝜃𝑥_𝑘𝛼_𝑘,𝑙,𝑓𝑥_𝑘𝛼_𝑘,𝑙∉ℱ_𝑘. (3) At the beginning of the algorithm, the filter is initialized by ℱ₀={(𝜃,𝑓)∈𝑅²∶𝜃≥𝑢}, where 𝑢 is a large positive number. Throughout the optimization the filter is then augmented in some iterations after the new iterate point 𝑥_𝑘+1 has been accepted. For this, the following updating formula is used: ℱ_𝑘+1=ℱ_𝑘𝜃,𝑓∈𝑅²∶𝜃≥1−𝛾_𝜃𝜃𝑥_𝑘,𝑓≥𝑓𝑥_𝑘−𝛾_𝑓𝜃𝑥_𝑘. (4)

Similar to the traditional strategy of the filter method, to avoid the convergence to a feasible point but not an optimal solution, we consider the following 𝑓-type switching condition: 𝑚_𝑘𝛼_𝑘,𝑙<0,−𝑚_𝑘𝛼_𝑘,𝑙>𝛿𝜃𝑥_𝑘^𝑠𝜃, (5) where 𝑚_𝑘(𝛼_𝑘,𝑙)=𝛼_𝑘,𝑙∇𝑓(𝑥_𝑘)^𝑇𝑑_𝑘, 𝛿>0, and 𝑠_𝜃∈(0,1).

When condition (5) holds, the step 𝑑_𝑘 is a descent direction for current objective function. Then, instead of insisting on (2), the Armijo-type reduction condition is employed as follows: 𝑓𝑥_𝑘𝛼_𝑘,𝑙≤𝑓𝑥_𝑘+𝜂𝑚_𝑘𝛼_𝑘,𝑙, (6) where 𝜂∈(0,(1/2)) is a fixed constant. If (5) and (6) hold for the accepted trial step size, we may call it an f-type point, and accordingly this iteration is called an 𝑓-type iteration. An 𝑓-type point should be accepted as 𝑥_𝑘+1 without updating of the filter; that is, ℱ_𝑘+1=ℱ_𝑘, while if a trial point 𝑥_𝑘(𝛼_𝑘,𝑙) does not satisfy the switching condition (5) but satisfies (2), we call it an ℎ-type point (or accordingly an h-type iteration). An ℎ-type point should be accepted as 𝑥_𝑘+1 with updating of the filter, and we denote the set of indices of those iterations where filter has been augmented by 𝒜⊂ℕ.

In our method, when ∇𝑓(𝑥_𝑘)^𝑇𝑑_𝑘<0, the line search is performed. In the situation where no admissible step size can be found, the method switches to a feasibility restoration phrase, whose purpose is to find a new iterate point that satisfies (2) and is also acceptable to the current filter by trying to decrease the constraint violation. In order to detect the situation where no admissible step size can be found and the restoration phase has to be invoked, we approximate a minimum desired step size using linear models of involved functions. For this, we define that 𝛼_𝑘^min=min𝛾_𝜃𝛾,−^𝑓𝜃𝑥_𝑘∇𝑓𝑥_𝑘^𝑇𝑑_𝑘𝛿𝜃𝑥,−^𝑘𝑠𝜃∇𝑓𝑥_𝑘^𝑇𝑑_𝑘. (7)

3. Description of the Algorithm

We will now proceed to propose a line search filter SQP method for inequality constrained optimization and formally describe the algorithmic details in this section. The proposed algorithm consists of inner loops, and in the next section we show that there is no endless loop and the method is implementable and globally convergent.

In our algorithm, motivated by [5–7], the quadratic subproblem (1) is replaced by the following problem: min∇𝑓𝑥_𝑘^𝑇1𝑑+2𝑑^𝑇𝐵_𝑘𝑑+𝑏_𝑘𝑡s.t.𝑔_𝑗𝑥_𝑘+∇𝑔_𝑗𝑥_𝑘^𝑇𝑥𝑑≤𝑡𝑗∈𝐼,0≤𝑡≤𝜃_𝑘, (8) where 𝑏_𝑘 is a positive parameter. Clearly, this subproblem is always consistent and is a convex programming if 𝐵_𝑘 is positive semidefinite.

Let (𝑑_𝑘,𝑡_𝑘)∈(𝑅^𝑛,𝑅¹) be the solution of subproblem (8). Then there exists a Lagrange multiplier (𝜆_𝑘,𝜐_𝑘,ℎ_𝑘)∈(𝑅^𝑚,𝑅¹,𝑅¹) satisfying the following KKT system: 𝑔_𝑗𝑥_𝑘+∇𝑔_𝑗𝑥_𝑘^𝑇𝑑_𝑘≤𝑡_𝑘𝑗∈𝐼,0≤𝑡_𝑘≤𝜃𝑥_𝑘,∇𝑓𝑥_𝑘+𝐵_𝑘𝑑_𝑘+_𝑗∈𝐼∇𝑔_𝑗𝑥_𝑘𝜆_𝑘𝑗𝑏=0,_𝑘−𝜆_𝑘1−𝜐_𝑘+ℎ_𝑘𝜆=0,_𝑘𝑗𝑔_𝑗𝑥_𝑘+∇𝑔_𝑗𝑥_𝑘^𝑇𝑑_𝑘−𝑡_𝑘,𝜐=0𝑗∈𝐼_𝑘𝑡_𝑘=0,ℎ_𝑘𝑡_𝑘−𝜃𝑥_𝑘𝜆=0,_𝑘≥0,𝜐_𝑘≥0,ℎ_𝑘≥0. (9)

Now, the algorithm for solving the problem (P) can be stated as follows.

Algorithm 1. Consider the following.

Step  1. Initialization: choose an initial point 𝑥₁∈𝑅^𝑛, an initial filter ℱ₁, an initial parameter 𝑏₁>0, a symmetric and positive definite matrix 𝐵₁∈𝑅^𝑛×𝑛, 𝛿>0, and 𝑠_𝜃∈(0,1). Choose 𝜌∈(0,(1/2)),𝛿₁,𝛿₂>0, and 𝛾_𝑓,𝛾_𝜃∈(0,1). Set 𝑘=1.

Step  2. Solve subproblem (8) with 𝑥_𝑘, 𝐵_𝑘, and 𝑏_𝑘 to obtain the solution (𝑑_𝑘,𝑡_𝑘), and let (𝜆_𝑘,𝑣_𝑘,ℎ_𝑘) be the Lagrange multiplier. Stop if 𝑑_𝑘=0 and 𝑡_𝑘=0.

Step  3. If 𝑑_𝑘=0 but 𝑡_𝑘≠0, set 𝑥_𝑘+1=𝑥_𝑘 and ℱ_𝑘+1=ℱ_𝑘; go to Step  6.

Step  4. If 𝑑_𝑘≠0 and ∇𝑓(𝑥_𝑘)^𝑇𝑑_𝑘≥0, go to Step  8. If 𝑑_𝑘≠0 and ∇𝑓(𝑥_𝑘)^𝑇𝑑_𝑘<0, using backtracking line search consider the following. Step  4.1. Initial line search: set 𝑙=1 and 𝛼_𝑘,𝑙=1. Step  4.2. Compute a new trial point. If the trial step size 𝛼_𝑘,𝑙<𝛼_𝑘^min, go to Step  8. Otherwise, compute new trial point 𝑥_𝑘(𝛼_𝑘,𝑙)=𝑥_𝑘+𝛼_𝑘,𝑙𝑑_𝑘. Check acceptability to the filter; if (𝜃(𝑥_𝑘(𝛼_𝑘,𝑙)),𝑓(𝑥_𝑘(𝛼_𝑘,𝑙)))∈ℱ_𝑘, reject the trial step size and go to Step  4.4. Step  4.3. Check sufficient decrease with respect to current iterate point. 4.3.1. Case I: condition (5) holds. If the Armijo condition (6) holds, accept the trial step (that is, an f-type iteration), and go to Step  4.5; otherwise, go to Step  4.4. 4.3.2. Case II: condition (5) does not hold. If (2) holds, accept the trial step (that is, an h-type iteration), and go to Step  4.5; otherwise, go to Step  4.4. Step  4.4. Choose a new trial size 𝛼_𝑘,𝑙+1=𝜌𝛼_𝑘,𝑙. Set 𝑙=𝑙+1, and go to Step  4.2. Step  4.5. Accept trial point. Set 𝛼_𝑘=𝛼_𝑘,𝑙 and 𝑥_𝑘+1=𝑥_𝑘(𝛼_𝑘).

Step  5. Augment filter if it is necessary. If 𝑘 is not an f-type iteration, augment the filter using (4); otherwise leave the filter unchanged; that is, set ℱ_𝑘+1=ℱ_𝑘.

Step  6. Update parameters. Compute 𝑎_𝑘 by 𝑎_𝑘=min𝑑_𝑘⁻¹,𝜆_𝑘1+𝛿₁. (10)

Set 𝑏_𝑘+1=𝑏_𝑘,if𝑏_𝑘≥𝑎_𝑘,𝑏_𝑘+𝛿₂,otherwise. (11)

Step  7. Update 𝐵_𝑘 to 𝐵_𝑘+1. Go to Step  2 with 𝑘 replaced by 𝑘+1.

Step  8. Obtain a new point 𝑥_𝑘+1 from a feasible restoration phrase. Set 𝑘=𝑘+1, and go to Step  2.

Remark 2. The mechanisms of the filter could ensure that (𝜃(𝑥_𝑘),𝑓(𝑥_𝑘))∉ℱ_𝑘.

Remark 3. The feasibility restoration phrase in Step  8 could be any iterative algorithm with the goal of finding a less infeasible point; for example, a nonlinear optimization algorithm is applied to minimise 𝜃 such as Algorithm B in [14].

4. Global Convergence of Algorithm

In this section, we will show the proposed algorithm is well defined and globally convergent under some mild conditions. Then throughout this paper, we always assume that the following assumptions hold.

Assumptions(A1) The sequence {𝑥_𝑘} remains in compacted subset 𝑋∈𝑅^𝑛.(A2) The functions 𝑓 and 𝑔_𝑗, 𝑗=1,2,…𝑚, are twice continuously differentiable.(A3) For any 𝑥∈𝑋, the vectors {∇𝑔_𝑗(𝑥),𝑗∈𝐼(𝑥)} are linearly independent, where 𝐼(𝑥)={𝑗∈𝐼∣𝑔_𝑗(𝑥)=𝜃(𝑥)}.(A4) There exist two constants 𝑞≥𝑝>0 such that the matrices sequence {𝐵_𝑘} satisfy 𝑝‖𝑑‖²≤𝑑^𝑇𝐵_𝑘𝑑≤𝑞‖𝑑‖² for all 𝑘 and 𝑑∈𝑅^𝑛.

The first lemma shows that there is no cycle between Step  3 and Step  6 in Algorithm 1.

Lemma 4. Let {𝑥_𝑘} and {𝑑_𝑘} be generated by Algorithm 1. If 𝑑_𝑘=0 and 𝑡_𝑘>0, then there exists a finite positive integer 𝑟 such that 𝑑_𝑘+𝑟≠0.

Proof. Suppose by contradiction that 𝑑_𝑘+𝑟=0 and 𝑡_𝑘+𝑟>0 for all 𝑟≥1 and some 𝑘∈ℕ. Then, we have 𝑥_𝑘+𝑟=𝑥_𝑘, and hence 𝑡_𝑘+𝑟=𝜃(𝑥_𝑘+𝑟)=𝜃(𝑥_𝑘)=𝑡_𝑘>0 and 𝑣_𝑘+𝑟=0. It follows from (9) and (10) that 𝑏_𝑘+𝑟=𝜆_𝑘+𝑟1−ℎ_𝑘+𝑟<𝜆_𝑘+𝑟1+𝛿₁=𝑎_𝑘+𝑟. (12) This together with (11) implies that 𝑏_𝑘+𝑟+1=𝑏_𝑘+𝑟+𝛿₂ for all 𝑟≥1. As a result, we have 𝑏_𝑘+𝑟𝜆⟶+∞,_𝑘+𝑟1=𝑏_𝑘+𝑟+ℎ_𝑘+𝑟⟶+∞as𝑟⟶+∞. (13) Because 𝑥_𝑘+𝑟=𝑥_𝑘 for all 𝑟≥1, we denote 𝑥_𝑘+𝑟=𝑥. So, we get 0<𝑡_𝑘=lim_𝑟→+∞𝑡_𝑘+𝑟=lim_𝑟→+∞𝜃𝑥_𝑘+𝑟=𝜃(𝑥),lim_𝑟→+∞𝑔_𝑖𝑥_𝑘+𝑟+∇𝑔_𝑖𝑥_𝑘+𝑟^𝑇𝑑_𝑘+𝑟=lim_𝑟→+∞𝑔_𝑖𝑥_𝑘+𝑟=𝑔_𝑖(𝑥)<𝜃(𝑥)=𝑡_𝑘,∀𝑖∉𝐼(𝑥). (14) Therefore, we claim that for all 𝑟≥1, 𝑔_𝑖𝑥_𝑘+𝑟+∇𝑔_𝑖𝑥_𝑘+𝑟^𝑇𝑑_𝑘+𝑟<𝑡_𝑘+𝑟,∀𝑖∉𝐼(𝑥). (15) We also have by (9) and (15) that for all 𝑟≥1, (𝜆_𝑘+𝑟)_𝑖=0, for all 𝑖∉𝐼(𝑥). This yields ∇𝑓𝑥_𝑘+𝑟+_{(𝑗∈𝐼𝑥)}∇𝑔_𝑗𝑥_𝑘+𝑟𝜆_{𝑘+𝑟𝑗}=0. (16) Without loss of generality, we assume that lim_𝑟→+∞𝜆_𝑘+𝑟/‖𝜆_𝑘+𝑟‖₁=𝜆. Dividing by ‖𝜆_𝑘+𝑟‖₁ in both sides of the last equality and taking limits as 𝑟→+∞, we obtain _{(𝑗∈𝐼𝑥)}𝜆_𝑗∇𝑔_𝑗(𝑥)=0,𝜆_𝑗≥0,𝑗∈𝐼(𝑥). (17) Moreover, there exists at least one 𝑗∈𝐼(𝑥) satisfying 𝜆_𝑗>0. This contradicts Assumption (A3).

The next lemma shows that the value of the parameter 𝑏_𝑘 increased only a finite number of times.

Lemma 5. Let {𝑥_𝑘} be an infinite sequence generated by Algorithm 1; then there exists an integer 𝑘₀ such that 𝑏_𝑘=𝑏_max for all 𝑘≥𝑘₀. In addition, 𝑏_max=max{𝑏_𝑘}.

Proof. The proof is by contradiction. Suppose that 𝑏_𝑘→+∞, as 𝑘→+∞. The step  6 of Algorithm 1 implies that inequality 𝑏_𝑘<𝑎_𝑘=min𝑑_𝑘⁻¹,𝜆_𝑘1+𝛿₁ (18) holds for infinitely many 𝑘. In addition, there exists a subsequence satisfying lim_{𝑘𝑖→∞}𝑑_𝑘𝑖=0,lim_{𝑘𝑖→∞}𝜆_𝑘𝑖1=+∞. (19) Without loss of generality, using Assumption (A1), we suppose that 𝑥_𝑘𝑖→𝑥 as 𝑘_𝑖→∞. In a way similar to the proof of Lemma 4, we can derive, for all sufficiently large 𝑘_𝑖, 𝑔_𝑗𝑥_𝑘𝑖+∇𝑔_𝑗𝑥_𝑘𝑖^𝑇𝑑_𝑘𝑖<𝑡_𝑘𝑖,∀𝑗∉𝐼(𝑥). (20) By (9) we have that for all 𝑘_𝑖 sufficiently large, (𝜆_𝑘𝑖)_𝑗=0,forall𝑗∉𝐼(𝑥), which yields ∇𝑓𝑥_𝑘𝑖+𝐵_𝑘𝑖𝑑_𝑘𝑖+_{(𝑗∈𝐼𝑥)}∇𝑔_𝑗𝑥_𝑘𝑖𝜆_𝑘𝑖𝑗=0. (21) We can also assume that lim_{𝑘𝑖→+∞}𝜆_𝑘𝑖/‖𝜆_𝑘𝑖‖₁=𝜆. Dividing by ‖𝜆_𝑘𝑖‖₁ in both sides of the last equality and taking limits as 𝑘_𝑖→+∞, then _{(𝑗∈𝐼𝑥)}𝜆_𝑗∇𝑔_𝑗(𝑥)=0,𝜆_𝑗≥0,𝑗∈𝐼(𝑥). (22) Furthermore, there exists at least one 𝑗∈𝐼(𝑥) satisfying 𝜆_𝑗>0. This contradicts Assumption (A3).
Because 𝑏_𝑘 is increased only finitely many times, it is easy to see that 𝑏_max=max{𝑏_𝑘}.

Lemma 6. The line search is actually performed; that is, our method can generate 𝑑_𝑘 such that ∇𝑓(𝑥_𝑘)^𝑇d_𝑘<0 at some iteration.

Proof. The proof is by contradiction. Suppose that 𝑑_𝑘 satisfying ∇𝑓(𝑥_𝑘)^𝑇𝑑_𝑘<0 cannot be generated and the current iterate point cannot be changed as Step  3; then the iterate points cannot be escaped from the feasibility restoration phrase. Therefore, all points of {𝑥_𝑘} are generated by the the feasibility restoration phrase, which follows |𝒜|=∞. Now we first prove that lim_𝑘→∞𝜃𝑥_𝑘=0. (23)
Consider an infinite subsequence {𝑘_𝑖} of 𝒜 with 𝜃(𝑥_𝑘𝑖)≥𝜀 for some 𝜀>0. At each iteration 𝑘_𝑖, (𝜃(𝑥_𝑘𝑖),𝑓(𝑥_𝑘𝑖)) is added to the filter which means that no other (𝜃,𝑓) can be added to the filter at a later stage within the area [𝜃(𝑥_𝑘𝑖)−𝛾_𝜃𝜃(𝑥_𝑘𝑖),𝜃(𝑥_𝑘𝑖)]×[𝑓(𝑥_𝑘𝑖)−𝛾_𝑓𝜃(𝑥_𝑘𝑖),𝑓(𝑥_𝑘𝑖)], and the area of each square is at least 𝛾_𝜃𝛾_𝑓𝜀². By Assumption (A1), we have 0≤𝜃(𝑥)≤𝜃^max and 𝑓^min≤𝑓(𝑥)≤𝑓^max, and then (𝜃,𝑓) associated with the filter are restricted to ℬ=[0,𝜃^max]×[𝑓^min,𝑓^max]. Hence, ℬ is completely covered by at most a finite number of such areas, which forms contradiction with the infinite subsequence {𝑘_𝑖} satisfying (3). This means that (23) is true.
We can see that (0,…,0,𝜃(𝑥_𝑘))^𝑇∈𝑅^𝑛+1 is a feasible point of QP subproblem (8); then we get ∇𝑓𝑥_𝑘^𝑇𝑑_𝑘+12𝑑_𝑘^𝑇𝐵_𝑘𝑑_𝑘+𝑏_𝑘𝑡_𝑘<𝑏_𝑘𝜃𝑥_𝑘. (24) Because our method does not stop at Step  2, it holds that (𝑑_𝑘,𝑡_𝑘)↛(0,0). By 0≤𝑡_𝑘≤𝜃(𝑥_𝑘), (23) implies that lim_𝑘→∞𝑡_𝑘=0. With the facts (𝑑_𝑘,𝑡_𝑘)↛(0,0) and lim_𝑘→∞𝑡_𝑘=0, we can assume that ‖𝑑_𝑘‖>𝜖₀ for some 𝜖₀>0. Let 𝜖₁=𝑝𝜖²₀/4𝑏_max; (23) implies that for 𝑘 sufficiently large enough it holds that 𝜃(𝑥_𝑘)<𝜖₁, and then 𝑏_𝑘𝜃(𝑥_𝑘)≤(𝑝𝑏_𝑘𝜖²₀/4𝑏_max)≤(1/4)(𝑑_𝑘)^𝑇𝐵_𝑘𝑑_𝑘; thus, ∇𝑓𝑥_𝑘^𝑇𝑑_𝑘1<−2𝑑_𝑘^𝑇𝐵_𝑘𝑑_𝑘−𝑏_𝑘𝑡_𝑘+𝑏_𝑘𝜃𝑥_𝑘1<−2𝑑_𝑘^𝑇𝐵_𝑘𝑑_𝑘+𝑏_𝑘𝜃𝑥_𝑘1≤−4𝑑_𝑘^𝑇𝐵_𝑘𝑑_𝑘≤−𝑝𝜖²₀4<0, (25) which is a contradiction.

Lemma 7. If 𝜃(𝑥_𝑘)=0, then 𝑚_𝑘(𝛼)<0 for all 𝛼∈(0,1]. In addition, Θ_𝑘=min{𝜃∶(𝜃,𝑓)∈ℱ_𝑘}>0 for all 𝑘.

Proof. If 𝜃(𝑥_𝑘)=0, then (0,…,0)^𝑇∈𝑅^𝑛+1 is a feasible point of subproblem (8), so we get 𝑚_𝑘(𝛼)𝛼=∇𝑓𝑥_𝑘^𝑇𝑑_𝑘1≤−2𝑑_𝑘^𝑇𝐵_𝑘𝑑_𝑘−𝑏_𝑘𝑡_𝑘<0. (26) For proof of second statement please see [11, Lemma  4].

Theorem 8. Suppose all stated assumptions hold. Then lim_𝑘→∞𝜃𝑥_𝑘=0. (27)

Proof. Please see [11, Theorem  1].

Lemma 9. If Algorithm 1 generates an infinite sequence {𝑥_𝑘}, then the sequence {𝑑_𝑘} is bounded; that is to say, there exists a constant 𝑀_𝑑>0, such that for all 𝑘≥0, 𝑑_𝑘≤𝑀_𝑑. (28)

Proof. We can see that (0,…,0,𝜃(𝑥_𝑘))^𝑇∈𝑅^𝑛+1 is a feasible point of QP subproblem (8), so we get ∇𝑓𝑥_𝑘^𝑇𝑑_𝑘+12𝑑_𝑘^𝑇𝐵_𝑘𝑑_𝑘+𝑏_𝑘𝑡_𝑘≤𝑏_𝑘𝜃𝑥_𝑘. (29) From Theorem 8, there exists a constant 𝑀_𝜃 such that 𝜃(𝑥_𝑘)≤𝑀_𝜃; thus, −∇𝑓𝑥_𝑘⋅𝑑_𝑘+12𝑝𝑑_𝑘²≤∇𝑓𝑥_𝑘^𝑇𝑑_𝑘+12𝑑_𝑘^𝑇𝐵_𝑘𝑑_𝑘+𝑏_𝑘𝑡_𝑘≤𝑏_𝑘𝜃𝑥_𝑘≤𝑏_max𝑀_𝜃, (30) which implies that {𝑑_𝑘} is bounded; then, there exists a constant 𝑀_𝑑>0 such that 𝑑_𝑘≤𝑀_𝑑. (31)

Lemma 10. If {𝑥_𝑘𝑖} is a subsequence of iterate points for which ‖𝑑_𝑘𝑖‖≥𝜖 with some constant 𝜖>0 independent of 𝑖, then there exist constants 𝜖₁,𝜖₂>0, such that 𝜃𝑥_𝑘𝑖≤𝜖₁⟹𝑚_𝑘𝑖(𝛼)≤−𝜖₂𝛼 (32) for all 𝑖 and 𝛼∈(0,1].

Proof. We can see that (0,…,0,𝜃(𝑥_𝑘𝑖))^𝑇∈𝑅^𝑛+1 is a feasible point of QP subproblem (8); then we get ∇𝑓𝑥_𝑘𝑖^𝑇𝑑_𝑘𝑖+12𝑑_𝑘𝑖^𝑇𝐵_𝑘𝑖𝑑_𝑘𝑖+𝑏_𝑘𝑖𝑡_𝑘𝑖<𝑏_𝑘𝑖𝜃𝑥_𝑘𝑖. (33) Let 𝜖₁=𝑝𝜖²/4𝑏_max; then 𝑏_𝑘𝑖𝜃(𝑥_𝑘𝑖)≤(𝑝𝑏_𝑘𝑖𝜖²/4𝑏_max)≤(1/4)(𝑑_𝑘𝑖)^𝑇𝐵_𝑘𝑖𝑑_𝑘𝑖; thus, 𝑚_𝑘𝑖(𝛼)𝛼=∇𝑓𝑥_𝑘𝑖^𝑇𝑑_𝑘𝑖1<−2𝑑_𝑘𝑖^𝑇𝐵_𝑘𝑖𝑑_𝑘𝑖−𝑏_𝑘𝑖𝑡_𝑘𝑖+𝑏_𝑘𝑖𝜃𝑥_𝑘𝑖1<−2𝑑_𝑘𝑖^𝑇𝐵_𝑘𝑖𝑑_𝑘𝑖+𝑏_𝑘𝑖𝜃𝑥_𝑘𝑖1≤−4𝑑_𝑘𝑖^𝑇𝐵_𝑘𝑖𝑑_𝑘𝑖≤−𝑝𝜖²4. (34) Define 𝜖₂=𝑝𝜖²/4; then it implies that 𝑚_𝑘𝑖(𝛼)≤−𝜖₂𝛼.

Lemma 11. Suppose that the filter is augmented only a finite number of times; that is, |𝒜|<∞. Then lim_𝑘→∞𝑑_𝑘=0. (35)

Proof. The proof of this lemma can be found in [11, Lemma  8]. There the proof is stated for slightly different circumstances, but it is easy to verify that it is still valid in our context.

Lemma 12. There exists some constant 𝐶_𝑓>0, such that 𝑓𝑥_𝑘+𝛼𝑑_𝑘−𝑓𝑥_𝑘−𝑚_𝑘(𝛼)≤𝐶_𝑓𝛼²𝑑_𝑘², (36) for 𝛼∈(0,1].

Proof. The inequality follows directly from the second order Taylor expansion.

Lemma 13. Let {𝑥_𝑘𝑖} be a subsequence with 𝑚_𝑘𝑖(𝛼)≤−𝛼𝜖₂ for some constant 𝜖₂>0 independent of 𝑘_𝑖 and for all 𝛼∈(0,1]. Then there exists some constant 𝛼>0 such that 𝑓𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖−𝑓𝑥_𝑘𝑖≤𝜂𝑚_𝑘𝑖(𝛼), (37) for all 𝑘_𝑖 and 𝛼≤𝛼.

Proof. Let 𝑀_𝑑 and 𝐶_𝑓 be the constants from Lemmas 9 and 12. It then follows from Lemma 12 for all 𝛼≤𝛼 with 𝛼=(1−𝜂)𝜖₂/𝐶_𝑓𝑀²_𝑑 that 𝑓𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖−𝑓𝑥_𝑘𝑖−𝑚_𝑘𝑖(𝛼)≤𝐶_𝑓𝛼²𝑑_𝑘𝑖²≤𝛼1−𝜂𝜖₂≤−1−𝜂𝑚_𝑘𝑖(𝛼). (38) That is, 𝑓𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖−𝑓𝑥_𝑘𝑖≤𝜂𝑚_𝑘𝑖(𝛼). (39)

Lemma 14. Let {𝑥_𝑘𝑖} be a subsequence with 𝑚_𝑘𝑖(𝛼)≤−𝛼𝜖₂ for some constant 𝜖₂>0 independent of 𝑘_𝑖 and for all 𝛼∈(0,1]. Then there exist constants 𝐶₁,𝐶(𝑥_𝑘𝑖)>0 so that 𝜃𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖,𝑓𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖∉ℱ_𝑘𝑖, (40) for all 𝑘_𝑖 and 𝛼≤min{𝐶₁,𝐶(𝑥_𝑘𝑖)}.

Proof. Let 𝑀_𝑑,𝐶_𝑓 be the constants from Lemmas 9 and 12. The mechanisms of Algorithm 1 ensure that 𝜃𝑥_𝑘𝑖,𝑓𝑥_𝑘𝑖∉ℱ_𝑘𝑖. (41)
Define ℱ_𝑘⁽¹⁾_𝑖={(𝜃,𝑓)∈ℱ_𝑘𝑖∣𝑓>𝑓(𝑥_𝑘𝑖)}, ℱ_𝑘⁽²⁾_𝑖=ℱ_𝑘𝑖⧵ℱ_𝑘⁽¹⁾_𝑖, and 𝐶₁=min{1,𝜖₂/(𝑀²_𝑑𝐶_𝑓)}. For 𝛼≤𝐶₁ we have 𝛼²≤(𝛼𝜖₂/𝑀²_𝑑𝐶_𝑓)≤(−𝑚_𝑘𝑖(𝛼)/𝐶_𝑓‖𝑑_𝑘𝑖‖²), or equivalently 𝑚_𝑘𝑖(𝛼)+𝐶_𝑓𝛼²𝑑_𝑘𝑖≤0, (42) and it follows with (36) that 𝑓𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖≤𝑓𝑥_𝑘𝑖. (43) So for 𝛼≤𝐶₁ we have (𝜃(𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖),𝑓(𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖))∉ℱ_𝑘⁽¹⁾_𝑖.
Let 𝜃_min=min{𝜃∣𝜃∈ℱ_𝑘⁽²⁾_𝑖}; from (41) we know that 𝜃(𝑥_𝑘𝑖)<𝜃_min. Therefore, there exists some constant 𝐶(𝑥_𝑘𝑖) such that for 𝛼≤𝐶(𝑥_𝑘𝑖), it holds that 𝜃(𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖)<𝜃_min. So for 𝛼≤𝐶(𝑥_𝑘𝑖) we have (𝜃(𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖),𝑓(𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖))∉ℱ_𝑘⁽²⁾_𝑖. Thereby for 𝛼<min{𝐶₁,𝐶(𝑥_𝑘𝑖)}, we have 𝜃𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖,𝑓𝑥_𝑘𝑖+𝛼𝑑_𝑘𝑖∉ℱ_𝑘⁽¹⁾_𝑖∪ℱ_𝑘⁽²⁾_𝑖=ℱ_𝑘𝑖. (44)

Lemma 15. Let {𝑥_𝑘𝑖} be a subsequence with ‖𝑑_𝑘𝑖‖≥𝜖 for some constant 𝜖>0 independent of 𝑘_𝑖. Then there exists 𝐾∈ℕ such that the filter is not augmented in iteration 𝐾; that is, 𝐾∉𝒜.

Proof. Since we have lim_𝑖→∞𝜃(𝑥_𝑘𝑖)=0 by Theorem 8, it follows from Lemma 10 that there exist constants 𝜖₁,𝜖₂>0 such that 𝜃𝑥_𝑘𝑖≤𝜖₁,𝑚_𝑘𝑖(𝛼)≤−𝛼𝜖₂, (45) for 𝑘_𝑖 sufficiently large and 𝛼∈(0,1]. Without loss of generality we can assume that (45) is valid for all 𝑘_𝑖.
We can now apply Lemmas 13 and 14 to obtain the constants 𝛼,𝐶₁,𝐶(𝑥_𝑘𝑖)>0. As lim_𝑖→∞𝜃(𝑥_𝑘𝑖)=0, choose a constant 𝐶₂ and a sufficiently large 𝐾∈{𝑘_𝑖} such that 𝐶(𝑥_𝐾)>𝐶₂𝜃(𝑥_𝐾)>0 and 𝜃𝑥_𝐾<min𝛼𝐶₂,𝐶₁𝐶₂,𝜌𝐶₂𝜖₂𝛿^{1/(𝑠𝜃−1)}. (46) We note that (46) implies that 𝛿𝜃𝑥_𝐾^𝑠𝜃𝜖₂<𝜌𝐶₂𝜃𝑥_𝐾, (47) as well as 𝐶₂𝜃𝑥_𝐾<min𝛼,𝐶₁. (48) Now define 𝛽_𝐾=𝐶₂𝜃𝑥_𝐾=min𝛼,𝐶₁,𝐶₂𝜃𝑥_𝐾. (49) Lemmas 13 and 14 then imply that a trial step size 𝛼_𝐾,𝑙≤𝛽_𝐾 satisfies 𝑓𝑥_𝐾𝛼_𝐾,𝑙≤𝑓𝑥_𝐾+𝜂𝑚_𝐾𝛼_𝐾,𝑙,𝜃𝑥_𝐾𝛼_𝐾,𝑙,𝑓𝑥_𝐾𝛼_𝐾,𝑙∉ℱ_𝐾. (50) If we now denote with 𝛼_𝐾,𝐿 the first step size satisfying (50), the backtracking line search procedure in Step  4 then implies that for 𝛼≥𝛼_𝐾,𝑙𝛼≥𝜌𝛽_𝐾=𝜌𝐶₂𝜃𝑥_𝐾>𝛿𝜃𝑥_𝐾^𝑠𝜃𝜖₂, (51) and therefore for 𝛼≥𝛼_𝐾,𝑙𝛿𝜃𝑥_𝐾^𝑠𝜃<𝛼𝜖₂≤−𝑚_𝐾(𝛼). (52) This means that 𝛼_𝐾,𝑙 and all previous trial step sizes are f-step sizes. Hence the method does not switch to feasibility restoration phrase in Step  4 for those trial step sizes. Therefore, 𝛼_𝐾,𝑙 is the accepted step size 𝛼_𝐾 indeed. Since it satisfies both the f-type switching conditions (5) and (6), the filter is not augmented in iteration K.

Lemma 16. Suppose that the filter is augmented infinite number of times; that is, |𝒜|=∞. Then lim_{𝑘→∞,𝑘∈𝒜}𝑑_𝑘=0. (53)

Proof. Suppose that lim_{𝑘→∞,𝑘∈𝒜}sup‖𝑑_𝑘‖>0. Then there exist a subsequence {𝑥_𝑘𝑗} of {𝑥_𝑘,𝑘∈𝒜} and a constant 𝜖>0, such that lim_𝑗→∞𝜃(𝑥_𝑘𝑗)=0 and ‖𝑑_𝑘𝑗‖≥𝜖 for all 𝑘_𝑗. Applying Lemma 15 to {𝑥_𝑘𝑗}, we see that there is an iteration 𝑘_𝑗, in which the filter is not augmented; that is to say, 𝑘_𝑗∉𝒜. This contradicts the choice of {𝑥_𝑘𝑗}, and then the claim follows.

Although 0≤𝑡_𝑘≤𝜃(𝑥_𝑘) and Theorem 8 can imply that lim_𝑘→∞𝑡_𝑘=0, the next lemma shows better result.

Lemma 17. Let {𝑥_𝑘𝑖} be a subsequence with 𝑑_𝑘𝑖→0; then 𝑡_𝑘𝑖=0 for 𝑘_𝑖 sufficiently large.

Proof. Consider that 𝑑_𝑘𝑖→0 together with Lemma 5 implies for 𝑘 sufficiently large 𝑏_𝑘𝑖≥𝑎_𝑘𝑖=min𝑑_𝑘𝑖⁻¹,𝜆_𝑘𝑖1+𝛿₁=𝜆_𝑘𝑖1+𝛿₁. (54) On the other hand, we have from (9) that 𝑏_𝑘𝑖=𝜆_𝑘𝑖1+𝑣_𝑘𝑖−ℎ_𝑘𝑖. (55) Thereby, it holds that 𝑣_𝑘𝑖>ℎ_𝑘𝑖+𝛿₁>0 for 𝑘_𝑖 sufficiently large. So we get 𝑡_𝑘𝑖=0 for 𝑘_𝑖 sufficiently large.

Theorem 18. Suppose that all stated assumptions hold; the outcome of applying Algorithm 1 is one of the following.(A) A KKT point of problem (P) is found (Algorithm 1 stops at Step  2).(B) All limit points are feasible, and there exists at least one limit point 𝑥^∗ of {𝑥_𝑘} which is a KKT point for the inequality constrained NLP (1); that is to say, lim_𝑘→∞𝜃𝑥_𝑘=0, (56) and there exists at least one subsequence 𝒦 such that lim_{𝑘→∞,𝑘∈𝒦}𝑑_𝑘=lim_{𝑘→∞,𝑘∈𝒦}𝑡_𝑘=0. (57)

Proof. (A) It is obviously true.
(B) Equation (56) follows from Theorem 8. In order to show (57), we now distinguish it into two cases:(i) if the filter is augmented only a finite number of times, from Lemmas 11 and 17 we know every limit point is a KKT point for the inequality constrained NLP (1), so the claim is true, (ii) if the filter is augmented infinite number of times, from Lemmas 16 and 17 it holds that lim_{𝑘→∞,𝑘∈𝒜}𝑑_𝑘=lim_{𝑘→∞,𝑘∈𝒜}𝑡_𝑘=0. (58) Let 𝒦=𝒜; the claim is also true.
In a word, if Algorithm 1 generates an infinite sequence of iterate {𝑥_𝑘}, all limit points are feasible, and there exists at least one limit point 𝑥^∗ of {𝑥_𝑘} which is a KKT point for the inequality constrained NLP (1).

5. Numerical Experiments

In this section, we carry out some typical numerical experiments based on Algorithm 1. In the whole process, the program is coded in MATLAB 7.0, and we use its optimization toolbox to solve the QP subproblem (8).(1) During the numerical experiments, updating of 𝐵_𝑘 is done by the following damped BFGS formula (see [27, Chapter 18]): 𝐵_𝑘+1=𝐵_𝑘−𝐵_𝑘𝑠_𝑘𝑠_𝑘^𝑇𝐵_𝑘𝑠_𝑘^𝑇𝐵_𝑘𝑠_𝑘+𝑟_𝑘𝑟_𝑘^𝑇𝑠_𝑘^𝑇𝑟_𝑘, (59) where 𝑠_𝑘=𝑥_𝑘+1−𝑥_𝑘, 𝑟_𝑘=𝜃_𝑘𝑦_𝑘+(1−𝜃_𝑘)𝐵_𝑘𝑠_𝑘, 𝑦_𝑘=∇_𝑥𝐿(𝑥_𝑘+1,𝜆_𝑘)−∇_𝑥𝐿(𝑥_𝑘,𝜆_𝑘) and 𝜃_𝑘=1if𝑠_𝑘^𝑇𝑦_𝑘≥0.2𝑠_𝑘^𝑇𝐵_𝑘𝑠_𝑘,0.8𝑠_𝑘^𝑇𝐵_𝑘𝑠_𝑘𝑠_𝑘^𝑇𝐵_𝑘𝑠_𝑘−𝑠_𝑘^𝑇𝑦_𝑘otherwise. (60)(2) The algorithm parameters are set as follows: 𝐵₁=𝐸_𝑛,𝑏₁=111,𝑠_𝜃=0.75,𝛿=1,𝛿₁=10,𝛿₂=11,𝜂=0.25,𝜌=0.5,𝛾_𝑓=𝛾_𝜃=0.01,𝑢=1000. (61)(3) The stop criteria are ‖𝑑^𝑘₀‖ sufficiently small, where 𝑑^𝑘₀=(𝑑^𝑇_𝑘,𝑡_𝑘)^𝑇. In particular, the stop criteria of Step  2 are changed to if𝑑^𝑘₀≤10⁻⁷,stop. (62)(4) Now we report the numerical results over a set of problems from [28], where no equality constraints are contained and an initial point is provided for each problem. In addition, we tested our algorithm in two cases: a feasible initial point is provided for each problem or an infeasible initial point is given. These results are listed in Tables 1 and 2, respectively. Because the quadratic subproblem (1) is modified but the filter technique is not used in [5, 7] and the filter technique is used but the quadratic subproblem (1) is not modified in [18, 23], for the sake of comparing equally, under the same initial points, we select examples tested both in [5, 7] and examples tested both in [18, 23] to do the comparisons. The comparison numerical results among our algorithm and two groups of methods are listed in Tables 3 and 4, respectively.

Table 1 

The detail information of the numerical results for feasible initial point.

Table 2 

The detail information of the numerical results for infeasible initial point.

Table 3 

Comparison of our algorithm with two methods in [5, 7].

Table 4 

Comparison of our algorithm with two methods in [18, 23].

In Tables 1, 2, 3, and 4, the notations mean as follows:(i)No.: the number of problems in [28], (ii)Pro.: the name of problems in [28],(iii)𝑛: the number of variables,(iv)𝑚: the number of inequality constraints,(v)NIT: the number of iterations,(vi)NOF: the number of evaluations of objective functions,(vii)NOG: the number of evaluations of objective function gradient,(viii)FV: the final value of the objective function,(ix)𝐹: it indicates the algorithm's failure on the problem,(x)FilterSQP1: our algorithm in this paper,(xi)JSQP: the algorithm proposed in [5],(xii)CSQP: the algorithm proposed in [7],(xiii)FilterSQP2: the algorithm proposed in [18],(xiv)FilterSQP3: the algorithm proposed in [23].

From Tables 1 and 2, we can see that our algorithm executes well for these problems taken from [28]. Whenever the initial point is feasible or not, the results are promising. From the computation efficiency in Tables 3 and 4, we should point out that our algorithm is competitive with some existed SQP methods in terms of the number of iterations, for example, [5, 7, 18, 23].

According to the forms of examples listed in these two groups of papers, we show the numerical results by two different representations in Tables 3 and 4. From Table 3, our algorithm is competitive with the first group of methods, that is, two methods in [5, 7]. Especially utilizing the algorithm in this paper, the number of iterations for problem hs001 (i.e., PBRT11) is much smaller than ones in other two algorithms. And the problem hs011 (i.e., QQRT12) converges to a good approximate optimal solution with small iterations, while algorithms CSQP and JSQP both fail on it. From Table 4, our algorithm is also competitive with the second group of methods, that is, two methods in [18, 23]. Although the filter technique is taken in these three algorithms, the quadratic subproblem (1) in [18, 23] may be inconsistent, while the subproblem is always consistent, and it only needs to solve one QP subproblem at each iteration in our algorithm, which is simpler and can be applied much more conveniently.

All results summarized in Tables 1–4 show that our algorithm is practical and effective.

6. Conclusion

In this paper, combining a modification strategy of QP subproblem and the filter technique, we present a line search filter SQP method for inequality constrained optimization. This method can start with any arbitrary initial point rather than a feasible initial point; it makes use of a backtracking line search procedure to generate step size and the efficiency of the filter technique to determine step acceptance; it only needs to solve one QP subproblem at each iteration, and the subproblem is always consistent. Under some mild conditions, the method is well defined, and the global convergence property is obtained. Many numerical experiments in Section 5 show that our algorithm is effective.

Acknowledgments

The author would like to thank the anonymous referees for the careful reading and helpful comments and suggestions, which led to an improved version of the original paper. Research is supported by the National Natural Science Foundation of China (no. 11226327 and no. 71102043) and Science & Technology Commission of Shanghai Municipality (no. 12510501700).