Research Article | Open Access
Zhong Jin, "A Globally Convergent Line Search Filter SQP Method for Inequality Constrained Optimization", Journal of Applied Mathematics, vol. 2013, Article ID 524539, 10 pages, 2013. https://doi.org/10.1155/2013/524539
A Globally Convergent Line Search Filter SQP Method for Inequality Constrained Optimization
A line search filter SQP method for inequality constrained optimization is presented. This method makes use of a backtracking line search procedure to generate step size and the efficiency of the filter technique to determine step acceptance. At each iteration, the subproblem is always consistent, and it only needs to solve one QP subproblem. Under some mild conditions, the global convergence property can be guaranteed. In the end, numerical experiments show that the method in this paper is effective.
In this paper, we consider the following nonlinear inequality constrained optimization problem: where , and are assumed to be continuously differentiable.
It is well known that the sequential quadratic programming (SQP) method is one of the most efficient methods to solve problem . Because of its superlinear convergence rate, it has been widely studied. See Boggs and Tolle  for an excellent literature survey. In SQP methods, at each iteration the search direction is generally obtained by solving the subproblem as follows: where is a symmetric positive definite matrix.
However, the previous QP subproblem has a serious shortcoming that constraints in (1) may be inconsistent. To overcome this disadvantage, much attention has been paid [2–7]. Burke and Han , Zhou , and J.-L. Zhang and X.-S. Zhang  modified the constraints in subproblem (1) to ensure that a revised QP subproblem is consistent, and their methods are globally convergent. Burke and Han's method is just a conceptual method and cannot be implementable practically, Zhou's method is based on the exact line search, and Zhang's method focuses on the inexact line search. Furthermore, Pantoja and Mayne  presented a modification of SQP algorithm by modifying both constraints and objective function in (1), and the search direction is obtained by solving a new QP subproblem which always has an optimal solution. In addition, Liu and Li  and Liu and Zeng  proposed SQP algorithms with cautious update criteria, which can be considered as modifications of the SQP algorithm given in .
In the previous methods [2–7], a penalty function is always used as a merit function to test the acceptability of the iterate points. However, as we all know, there are several difficulties associated with the use of penalty function and in particular with the choice of the penalty parameter. Too low a choice may result in the loss of an optimal solution; on the other hand, too large a choice damps out the effect of the objective function. Therefore, filter method was first introduced in trust region SQP method for constrained nonlinear optimization problems by Fletcher and Leyffer , offering an alternative to merit functions, as a tool to guarantee global convergence. Subsequently, global convergences of filter SQP methods were established by Fletcher et al. [9, 10]. Furthermore, Wächter and Biegler [11, 12] presented line search filter methods for nonlinear equality constrained programming, and the global convergence and local convergence were given. The promising numerical results of filter methods have led to a growing interest in filter methods in recent years [13–26], in which trust region strategy is used in [15, 26], and line search technique is taken in others. It is noteworthy that, based on [2–4] and line search filter technique, Su and Che  and Su  presented modified SQP filter methods for inequality constrained optimization. In Su's methods, the subproblem is always consistent, and the global convergence property is obtained.
In this paper, derived from [5–7] and the filter technique, especially the filter line search technique provided by Wächter and Biegler [11, 12], we propose a line search filter SQP method for inequality constrained optimization. This method has the following merits: it avoids using the penalty function to guarantee global convergence and has no requirement on initial point; at each iterate point the subproblem is always consistent, and it only needs to solve one QP subproblem; under some mild conditions, we prove that the algorithm either terminates at a Karush-Kuhn-Tucker (KKT) point within finite steps or generates an infinite sequence whose at least one cluster is a KKT point; in the end, numerical experiments show that our method is effective.
This paper is organized as follows. In the next section, the structure of line search filter technique is discussed. The algorithm is put forward in Section 3, and the global convergence theory of the algorithm is presented in Section 4. The numerical results for some typical examples are listed in Section 5. Finally in Section 6, a brief discussion on the proposed algorithm is given.
2. The Structure of Line Search Filter Technique
In this section, we will present the structure of line search filter technique and some related strategies. Instead of combing the objective function and constraint violation into a single function, filter methods view nonlinear optimization as a biobjective optimization problem that minimizes the objective function and the constraint violation. Now we give the structure of line search filter technique.
In the line search filter technique, after a search direction has been computed, a step size is determined in order to obtain the trial iteration . More precisely, for fixed constants , a trial step size provides sufficient reduction with respect to the current point if where is a constraint violation function. Similar to , let and .
For the sake of a simplified notation, the filter is defined in this paper not as a list but as a set containing all pairs that are prohibited in iteration . We say that a trial point is acceptable to the filter if its pair does not lie in the taboo region, that is, if At the beginning of the algorithm, the filter is initialized by , where is a large positive number. Throughout the optimization the filter is then augmented in some iterations after the new iterate point has been accepted. For this, the following updating formula is used:
Similar to the traditional strategy of the filter method, to avoid the convergence to a feasible point but not an optimal solution, we consider the following -type switching condition: where , , and .
When condition (5) holds, the step is a descent direction for current objective function. Then, instead of insisting on (2), the Armijo-type reduction condition is employed as follows: where is a fixed constant. If (5) and (6) hold for the accepted trial step size, we may call it an f-type point, and accordingly this iteration is called an -type iteration. An -type point should be accepted as without updating of the filter; that is, , while if a trial point does not satisfy the switching condition (5) but satisfies (2), we call it an -type point (or accordingly an h-type iteration). An -type point should be accepted as with updating of the filter, and we denote the set of indices of those iterations where filter has been augmented by .
In our method, when , the line search is performed. In the situation where no admissible step size can be found, the method switches to a feasibility restoration phrase, whose purpose is to find a new iterate point that satisfies (2) and is also acceptable to the current filter by trying to decrease the constraint violation. In order to detect the situation where no admissible step size can be found and the restoration phase has to be invoked, we approximate a minimum desired step size using linear models of involved functions. For this, we define that
3. Description of the Algorithm
We will now proceed to propose a line search filter SQP method for inequality constrained optimization and formally describe the algorithmic details in this section. The proposed algorithm consists of inner loops, and in the next section we show that there is no endless loop and the method is implementable and globally convergent.
In our algorithm, motivated by [5–7], the quadratic subproblem (1) is replaced by the following problem: where is a positive parameter. Clearly, this subproblem is always consistent and is a convex programming if is positive semidefinite.
Let be the solution of subproblem (8). Then there exists a Lagrange multiplier satisfying the following KKT system:
Now, the algorithm for solving the problem can be stated as follows.
Algorithm 1. Consider the following.
Step 1. Initialization: choose an initial point , an initial filter , an initial parameter , a symmetric and positive definite matrix , , and . Choose , and . Set .
Step 2. Solve subproblem (8) with , , and to obtain the solution , and let be the Lagrange multiplier. Stop if and .
Step 3. If but , set and ; go to Step 6.
Step 4. If and , go to Step 8. If and , using backtracking line search consider the following. Step 4.1. Initial line search: set and . Step 4.2. Compute a new trial point. If the trial step size , go to Step 8. Otherwise, compute new trial point . Check acceptability to the filter; if , reject the trial step size and go to Step 4.4. Step 4.3. Check sufficient decrease with respect to current iterate point. 4.3.1. Case I: condition (5) holds. If the Armijo condition (6) holds, accept the trial step (that is, an f-type iteration), and go to Step 4.5; otherwise, go to Step 4.4. 4.3.2. Case II: condition (5) does not hold. If (2) holds, accept the trial step (that is, an h-type iteration), and go to Step 4.5; otherwise, go to Step 4.4. Step 4.4. Choose a new trial size . Set , and go to Step 4.2. Step 4.5. Accept trial point. Set and .
Step 5. Augment filter if it is necessary. If is not an f-type iteration, augment the filter using (4); otherwise leave the filter unchanged; that is, set .
Step 6. Update parameters. Compute by
Step 7. Update to . Go to Step 2 with replaced by .
Step 8. Obtain a new point from a feasible restoration phrase. Set , and go to Step 2.
Remark 2. The mechanisms of the filter could ensure that .
Remark 3. The feasibility restoration phrase in Step 8 could be any iterative algorithm with the goal of finding a less infeasible point; for example, a nonlinear optimization algorithm is applied to minimise such as Algorithm B in .
4. Global Convergence of Algorithm
In this section, we will show the proposed algorithm is well defined and globally convergent under some mild conditions. Then throughout this paper, we always assume that the following assumptions hold.
Assumptions(A1) The sequence remains in compacted subset .(A2) The functions and , , are twice continuously differentiable.(A3) For any , the vectors are linearly independent, where .(A4) There exist two constants such that the matrices sequence satisfy for all and .
The first lemma shows that there is no cycle between Step 3 and Step 6 in Algorithm 1.
Lemma 4. Let and be generated by Algorithm 1. If and , then there exists a finite positive integer such that .
Proof. Suppose by contradiction that and for all and some . Then, we have , and hence and . It follows from (9) and (10) that This together with (11) implies that for all . As a result, we have Because for all , we denote . So, we get Therefore, we claim that for all , We also have by (9) and (15) that for all , , for all . This yields Without loss of generality, we assume that . Dividing by in both sides of the last equality and taking limits as , we obtain Moreover, there exists at least one satisfying . This contradicts Assumption (A3).
The next lemma shows that the value of the parameter increased only a finite number of times.
Lemma 5. Let be an infinite sequence generated by Algorithm 1; then there exists an integer such that for all . In addition, .
Proof. The proof is by contradiction. Suppose that , as . The step 6 of Algorithm 1 implies that inequality
holds for infinitely many . In addition, there exists a subsequence satisfying
Without loss of generality, using Assumption (A1), we suppose that as . In a way similar to the proof of Lemma 4, we can derive, for all sufficiently large ,
By (9) we have that for all sufficiently large, , which yields
We can also assume that . Dividing by in both sides of the last equality and taking limits as , then
Furthermore, there exists at least one satisfying . This contradicts Assumption (A3).
Because is increased only finitely many times, it is easy to see that .
Lemma 6. The line search is actually performed; that is, our method can generate such that at some iteration.
Proof. The proof is by contradiction. Suppose that satisfying cannot be generated and the current iterate point cannot be changed as Step 3; then the iterate points cannot be escaped from the feasibility restoration phrase. Therefore, all points of are generated by the the feasibility restoration phrase, which follows . Now we first prove that
Consider an infinite subsequence of with for some . At each iteration , is added to the filter which means that no other can be added to the filter at a later stage within the area , and the area of each square is at least . By Assumption (A1), we have and , and then associated with the filter are restricted to . Hence, is completely covered by at most a finite number of such areas, which forms contradiction with the infinite subsequence satisfying (3). This means that (23) is true.
We can see that is a feasible point of QP subproblem (8); then we get Because our method does not stop at Step 2, it holds that . By , (23) implies that . With the facts and , we can assume that for some . Let ; (23) implies that for sufficiently large enough it holds that , and then ; thus, which is a contradiction.
Lemma 7. If , then for all . In addition, for all .
Theorem 8. Suppose all stated assumptions hold. Then
Proof. Please see [11, Theorem 1].
Lemma 9. If Algorithm 1 generates an infinite sequence , then the sequence is bounded; that is to say, there exists a constant , such that for all ,
Proof. We can see that is a feasible point of QP subproblem (8), so we get From Theorem 8, there exists a constant such that ; thus, which implies that is bounded; then, there exists a constant such that
Lemma 10. If is a subsequence of iterate points for which with some constant independent of , then there exist constants , such that for all and .
Proof. We can see that is a feasible point of QP subproblem (8); then we get Let ; then ; thus, Define ; then it implies that .
Lemma 11. Suppose that the filter is augmented only a finite number of times; that is, . Then
Proof. The proof of this lemma can be found in [11, Lemma 8]. There the proof is stated for slightly different circumstances, but it is easy to verify that it is still valid in our context.
Lemma 12. There exists some constant , such that for .
Proof. The inequality follows directly from the second order Taylor expansion.
Lemma 13. Let be a subsequence with for some constant independent of and for all . Then there exists some constant such that for all and .
Lemma 14. Let be a subsequence with for some constant independent of and for all . Then there exist constants so that for all and .
Proof. Let be the constants from Lemmas 9 and 12. The mechanisms of Algorithm 1 ensure that
Define , , and . For we have , or equivalently and it follows with (36) that So for we have .
Let ; from (41) we know that . Therefore, there exists some constant such that for , it holds that . So for we have . Thereby for , we have
Lemma 15. Let be a subsequence with for some constant independent of . Then there exists such that the filter is not augmented in iteration ; that is, .
Proof. Since we have by Theorem 8, it follows from Lemma 10 that there exist constants such that
for sufficiently large and . Without loss of generality we can assume that (45) is valid for all .
We can now apply Lemmas 13 and 14 to obtain the constants . As , choose a constant and a sufficiently large such that and We note that (46) implies that as well as Now define Lemmas 13 and 14 then imply that a trial step size satisfies If we now denote with the first step size satisfying (50), the backtracking line search procedure in Step 4 then implies that for and therefore for This means that and all previous trial step sizes are f-step sizes. Hence the method does not switch to feasibility restoration phrase in Step 4 for those trial step sizes. Therefore, is the accepted step size indeed. Since it satisfies both the f-type switching conditions (5) and (6), the filter is not augmented in iteration K.
Lemma 16. Suppose that the filter is augmented infinite number of times; that is, . Then
Proof. Suppose that . Then there exist a subsequence of and a constant , such that and for all . Applying Lemma 15 to , we see that there is an iteration , in which the filter is not augmented; that is to say, . This contradicts the choice of , and then the claim follows.
Although and Theorem 8 can imply that , the next lemma shows better result.
Lemma 17. Let be a subsequence with ; then for sufficiently large.
Theorem 18. Suppose that all stated assumptions hold; the outcome of applying Algorithm 1 is one of the following.(A) A KKT point of problem is found (Algorithm 1 stops at Step 2).(B) All limit points are feasible, and there exists at least one limit point of which is a KKT point for the inequality constrained NLP (1); that is to say, and there exists at least one subsequence such that
Proof. (A) It is obviously true.
(B) Equation (56) follows from Theorem 8. In order to show (57), we now distinguish it into two cases:(i) if the filter is augmented only a finite number of times, from Lemmas 11 and 17 we know every limit point is a KKT point for the inequality constrained NLP (1), so the claim is true, (ii) if the filter is augmented infinite number of times, from Lemmas 16 and 17 it holds that Let ; the claim is also true.
In a word, if Algorithm 1 generates an infinite sequence of iterate , all limit points are feasible, and there exists at least one limit point of which is a KKT point for the inequality constrained NLP (1).
5. Numerical Experiments
In this section, we carry out some typical numerical experiments based on Algorithm 1. In the whole process, the program is coded in MATLAB 7.0, and we use its optimization toolbox to solve the QP subproblem (8).(1) During the numerical experiments, updating of is done by the following damped BFGS formula (see [27, Chapter 18]): where , , and (2) The algorithm parameters are set as follows: (3) The stop criteria are sufficiently small, where . In particular, the stop criteria of Step 2 are changed to (4) Now we report the numerical results over a set of problems from , where no equality constraints are contained and an initial point is provided for each problem. In addition, we tested our algorithm in two cases: a feasible initial point is provided for each problem or an infeasible initial point is given. These results are listed in Tables 1 and 2, respectively. Because the quadratic subproblem (1) is modified but the filter technique is not used in [5, 7] and the filter technique is used but the quadratic subproblem (1) is not modified in [18, 23], for the sake of comparing equally, under the same initial points, we select examples tested both in [5, 7] and examples tested both in [18, 23] to do the comparisons. The comparison numerical results among our algorithm and two groups of methods are listed in Tables 3 and 4, respectively.
In Tables 1, 2, 3, and 4, the notations mean as follows:(i)No.: the number of problems in , (ii)Pro.: the name of problems in ,(iii): the number of variables,(iv): the number of inequality constraints,(v)NIT: the number of iterations,(vi)NOF: the number of evaluations of objective functions,(vii)NOG: the number of evaluations of objective function gradient,(viii)FV: the final value of the objective function,(ix): it indicates the algorithm's failure on the problem,(x)FilterSQP1: our algorithm in this paper,(xi)JSQP: the algorithm proposed in ,(xii)CSQP: the algorithm proposed in ,(xiii)FilterSQP2: the algorithm proposed in ,(xiv)FilterSQP3: the algorithm proposed in .
From Tables 1 and 2, we can see that our algorithm executes well for these problems taken from . Whenever the initial point is feasible or not, the results are promising. From the computation efficiency in Tables 3 and 4, we should point out that our algorithm is competitive with some existed SQP methods in terms of the number of iterations, for example, [5, 7, 18, 23].
According to the forms of examples listed in these two groups of papers, we show the numerical results by two different representations in Tables 3 and 4. From Table 3, our algorithm is competitive with the first group of methods, that is, two methods in [5, 7]. Especially utilizing the algorithm in this paper, the number of iterations for problem hs001 (i.e., PBRT11) is much smaller than ones in other two algorithms. And the problem hs011 (i.e., QQRT12) converges to a good approximate optimal solution with small iterations, while algorithms CSQP and JSQP both fail on it. From Table 4, our algorithm is also competitive with the second group of methods, that is, two methods in [18, 23]. Although the filter technique is taken in these three algorithms, the quadratic subproblem (1) in [18, 23] may be inconsistent, while the subproblem is always consistent, and it only needs to solve one QP subproblem at each iteration in our algorithm, which is simpler and can be applied much more conveniently.
In this paper, combining a modification strategy of QP subproblem and the filter technique, we present a line search filter SQP method for inequality constrained optimization. This method can start with any arbitrary initial point rather than a feasible initial point; it makes use of a backtracking line search procedure to generate step size and the efficiency of the filter technique to determine step acceptance; it only needs to solve one QP subproblem at each iteration, and the subproblem is always consistent. Under some mild conditions, the method is well defined, and the global convergence property is obtained. Many numerical experiments in Section 5 show that our algorithm is effective.
The author would like to thank the anonymous referees for the careful reading and helpful comments and suggestions, which led to an improved version of the original paper. Research is supported by the National Natural Science Foundation of China (no. 11226327 and no. 71102043) and Science & Technology Commission of Shanghai Municipality (no. 12510501700).
- P. T. Boggs and J. W. Tolle, “Sequential quadratic programming,” in Acta Numerica, pp. 1–51, Cambridge University Press, Cambridge, UK, 1995.
- J. V. Burke and S.-P. Han, “A robust sequential quadratic programming method,” Mathematical Programming, vol. 43, no. 3, pp. 277–303, 1989.
- G. Zhou, “A modified SQP method and its global convergence,” Journal of Global Optimization, vol. 11, no. 2, pp. 193–205, 1997.
- J.-L. Zhang and X.-S. Zhang, “A modified SQP method with nonmonotone linesearch technique,” Journal of Global Optimization, vol. 21, no. 2, pp. 201–218, 2001.
- J. F. A. D. O. Pantoja and D. Q. Mayne, “Exact penalty function algorithm with simple updating of the penalty parameter,” Journal of Optimization Theory and Applications, vol. 69, no. 3, pp. 441–467, 1991.
- T. Liu and D. Li, “Convergence of the BFGS-SQP method for degenerate problems,” Numerical Functional Analysis and Optimization, vol. 28, no. 7-8, pp. 927–944, 2007.
- T.-w. Liu and J.-p. Zeng, “An SQP algorithm with cautious updating criteria for nonlinear degenerate problems,” Acta Mathematicae Applicatae Sinica, vol. 25, no. 1, pp. 33–42, 2009.
- R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Mathematical Programming, vol. 91, no. 2, pp. 239–269, 2002.
- R. Fletcher, S. Leyffer, and P. L. Toint, “On the global convergence of a filter-SQP algorithm,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 44–59, 2002.
- R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A. Wächter, “Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 635–660, 2002.
- A. Wächter and L. T. Biegler, “Line search filter methods for nonlinear programming: motivation and global convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 1–31, 2005.
- A. Wächter and L. T. Biegler, “Line search filter methods for nonlinear programming: local convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 32–48, 2005.
- K. Su and J. Che, “A modified SQP-filter method and its global convergence,” Applied Mathematics and Computation, vol. 194, no. 1, pp. 92–101, 2007.
- K. Su, “A globally and superlinearly convergent modified SQP-filter method,” Journal of Global Optimization, vol. 41, no. 2, pp. 203–217, 2008.
- C. Shen, W. Xue, and X. Chen, “Global convergence of a robust filter SQP algorithm,” European Journal of Operational Research, vol. 206, no. 1, pp. 34–45, 2010.
- C. Shen, W. Xue, and D. Pu, “Global convergence of a tri-dimensional filter SQP algorithm based on the line search method,” Applied Numerical Mathematics, vol. 59, no. 2, pp. 235–250, 2009.
- W. Xue, C. Shen, and D. Pu, “A penalty-function-free line search SQP method for nonlinear programming,” Journal of Computational and Applied Mathematics, vol. 228, no. 1, pp. 313–325, 2009.
- X. Wang, Z. Zhu, S. Zuo, and Q. Huang, “An SQP-filter method for inequality constrained optimization and its global convergence,” Applied Mathematics and Computation, vol. 217, no. 24, pp. 10224–10230, 2011.
- C. Gu, “Nonmonotone filter DQMM method for the system of nonlinear equations,” Applied Mathematics and Computation, vol. 217, no. 22, pp. 9351–9357, 2011.
- C. Gu and D. Zhu, “A secant algorithm with line search filter method for nonlinear optimization,” Applied Mathematical Modelling, vol. 35, no. 2, pp. 879–894, 2011.
- C. Gu and D. Zhu, “A filter algorithm for nonlinear systems of equalities and inequalities,” Applied Mathematics and Computation, vol. 218, no. 20, pp. 10289–10298, 2012.
- Z. Wang and D. Zhu, “A reduced Hessian algorithm with line search filter method for nonlinear programming,” Applied Mathematics and Computation, vol. 217, no. 19, pp. 7679–7691, 2011.
- X. Zhu and D. Pu, “Sequential quadratic programming with a flexible step acceptance strategy,” Applied Mathematical Modelling, vol. 36, no. 9, pp. 3992–4002, 2012.
- X. Zhu and D. Pu, “A line search filter algorithm with inexact step computations for equality constrained optimization,” Applied Numerical Mathematics, vol. 62, no. 3, pp. 212–223, 2012.
- X. Zhu and D. Pu, “A restoration-free filter SQP algorithm for equality constrained optimization,” Applied Mathematics and Computation, vol. 219, no. 11, pp. 6016–6029, 2013.
- X. Zhang, Z. Liu, and S. Liu, “A trust region SQP-filter method for nonlinear second-order cone programming,” Computers & Mathematics with Applications, vol. 63, no. 12, pp. 1569–1576, 2012.
- J. Nocedal and S. J. Wright, Numerical Optimization, Springer Series in Operations Research, Springer, New York, NY, USA, 1999.
- W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, vol. 187 of Lecture Notes in Economics and Mathematical Systems, Springer-Verlag, Berlin, Germany, 1981.
Copyright © 2013 Zhong Jin. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.