Research Article  Open Access
Ke Su, Wei Liu, Xiaoli Lu, "Global Convergence of a New Nonmonotone Filter Method for Equality Constrained Optimization", Journal of Applied Mathematics, vol. 2013, Article ID 976509, 9 pages, 2013. https://doi.org/10.1155/2013/976509
Global Convergence of a New Nonmonotone Filter Method for Equality Constrained Optimization
Abstract
A new nonmonotone filter trust region method is introduced for solving optimization problems with equality constraints. This method directly uses the dominated area of the filter as an acceptability criterion for trial points and allows the dominated area decreasing nonmonotonically. Compared with the filtertype method, our method has more flexible criteria and can avoid Maratos effect in a certain degree. Under reasonable assumptions, we prove that the given algorithm is globally convergent to a first order stationary point for all possible choices of the starting point. Numerical tests are presented to show the effectiveness of the proposed algorithm.
1. Introduction
We analyze an algorithm for solving optimization problems where a smooth objective function is to be minimized subject to smooth nonlinear equality constraints. More formally, we consider the problem, where , the functions and are all twice continuously differentiable. For convenience, let and , and refers to , to , to and to , and so forth.
There are many trust region methods for equality constrained nonlinear programming , for example, Byrd et al. [1], Dennis Jr. et al. [2] and Powell and Yuan [3], but in these works, a penalty or augmented Lagrange function is always used to test the acceptability of the iterates. However, there are several difficulties associated with the use of penalty function, and in particular the choice of the penalty parameter. Hence, in 2002, Fletcher and Leyffer [4] proposed a class of filter method, which does not require any penalty parameter and has promising numerical results. Consequently, filter technique has been employed to many approaches, for instance, SLP methods [5], SQP methods [6–8], interior point approaches [9], bundle techniques [10], and so on.
Filter technique, in fact, exhibits a certain degree of nonmonotonicity. The nonmonotone technique was proposed by Grippo et al. in 1986 [11] and combined with many other methods. M. Ulbrich and S. Ulbrich [12] proposed a class of penaltyfunctionfree nonmonotone trust region methods for nonlinear equality constrained optimization without filter technique. Su and Pu [13] introduced a nonmonotone trust region method which used the nonmonotone technique in the traditional filter criteria. Su and Yu [14] presented a nonmonotone method without penalty function or filter. Gould and Toint [15] directly used the dominated area of the filter as an acceptability criteria for trial points and obtained the global convergence properties. We refer the reader [16–18] for some works about this issue.
Motivated by the ideas and methods above, we propose a modified nonmonotone filter trust region method for solving problem . Similar to the ByrdOmojokun class of algorithms, each step is decomposed into the sum of two distinct components, a quasinormal step and a tangential step. The main contribution of our paper is to employ the nonmonotone idea to the dominated area of the filter so that the new and more flexible criteria is given, which is different from that of Gould and Toint [15] and Su and Pu [13]. Under usual assumptions, we prove that the given algorithm is globally convergent to first order stationary points.
This paper is organized as follows. In Section 2, we introduce the fraction of Cauchy decrease and the composite SQP step. The new nonmonotone filter technique is given in Section 3. In Section 4, we propose the new nonmonotone filter method and present the global convergence properties in Section 5. Some numerical results are reported in the last section.
2. The Fraction of Cauchy Decrease and the Composite SQP Step
Consider the following unconstraint minimization optimization problem: where is a continuously differentiable function. A trust region algorithm for solving the above problem is an iterate procedure that computes a trial step as an approximate solution to the following subproblems: where is the Hessian matrix or an approximate to it and is a given trust region radius.
To assure the global convergence, the step is required only to satisfy a fraction of Cauchy decrease condition. This means that must predict via the quadratic model function at least as much as a fraction of the decreased given by the Cauchy step on ; that is, there exists a constant fixed across all iterations, such that where is the steepest descent step for inside the trust region.
Lemma 1. If the trial step satisfies a fraction of Cauchy decrease condition, then
Proof (see Powell [19] for the proof). Now, we turn to explain the composite SQP step. Given an approximate estimate of the solution at th iteration, following Dennis Jr. et al. [2] and M. Ulbrich and S. Ulbrich [12], we obtain the trial step by computing a quasinormal step and a tangential step . The purpose of the quasinormal step is to improve feasibility. To improve optimality, we seek in the tangential space of the linearized constraints in such a way that it provides sufficient decrease for a quadratic model of the objective function . Let , where is a symmetric approximation of .
is the solution to the subproblem
where is a trust region radius and . In order to improve the value of the objective function, we solve the following subproblem to get
Then we get the current trial step . Let , where and denote a matrix whose columns form a basis of the null space of . We refer to [2] for a more detailed discussion of this issue.
In usual way that impose a trust region in stepdecomposition methods, the quasinormal step and the tangential step are required to satisfy
where . Here, to simplify the proof, we only impose a trust region on and , which is natural.
Note that is the reduced gradient of in terms of the representation of the tangential step:
Define
Then the first order necessary optimality conditions (KarushKuhnTucker or KKT conditions) at a local solution of problem can be written as
3. A New Nonmonotone Filter Technique
In filter method, originally proposed by Fletcher and Leyffer [4], the acceptability of iterates is determined by comparing the value of constraint violation and the objective function with previous iterates collected in a filter. Define the violation function by , it is easy to see that if and only if is a feasible point, so a trial point should reduce either the value of constraint violation or that of the objective function .
In the process of the algorithm, we need to decide whether the trial point is any better than as an approximate solution to the problem . If we decide that this is the case, we say that the iteration is successful and choose as the next iterate. Let us denote by the set of all successful iterations, that is,
In traditional filter method, a point is called acceptable to the filter if and only if where , denotes the filter set. Define
A trial point is accepted if and only if .
Now, similar to the idea of Gould and Toint [15], we give a new modified nonmonotone filter technique. For any pair, define an area that represents its contribution to the area of , we hope this contribution is positive; that is, the area of is increasing. For convenience, we partition the right halfplane into four different regions (see Figure 1). Define to be the complement of . Let
These four parts are(1)the dominated part of the filter: ;(2)the undominated part of lower left corner of the half plane: (3)the undominated upper left corner: ;(4)the undominated lower right corner: .
Consider the trial point , if the filter is empty, then define its contribution to the area of the filter by where is a constant. If the filter is not empty, then define the contribution of to the area of the filter by four different formulae.
If , assume
If , assume
If , assume
If , assume where , and .
Figure 2 illustrate the corresponding areas in the filter. Horizontally dashed surfaces indicate a positive contribution and vertically dashed ones a negative contribution. Note that is a continuous function of .
Next, we should consider the updating of the filter. If , then
If , then
We now return to the question of deciding whether a trial point is acceptable for the filter or not. We will insist that this is a necessary condition for the iteration to be successful in the sense that . If we consider an iterate , there must exist a predecessor iteration such that . Under the monotonic situation, a trial point would be accepted whenever it results in an sufficient increase in the dominated area of the filter, that means would be accepted whenever where , is a constant. Under the nonmonotonic situation, we relax condition (23) to be where , is some reference iteration for , , , .
Compared to condition (2.21) [15], our condition (24) is more flexible if is negative.
According to condition (24), it is possible to accept even though it may be dominated. Then will be accepted if either (23) or (24) holds.
4. The New Nonmonotone Filter Trust Region Algorithm
Our algorithm is based on the usual trust region technique; define the predict reduction for the function to be and the actual reduction
Moreover, let , if there exists a nonzero constant such that and condition (23) and (24) hold, the trial point will be called acceptable. Then the next trial point is obtained, and for its feasibility, we consider the condition is true or not, where and are all positive constants, if it is not true, then turn to the feasibility restoration phase and define A formal description of the algorithm is given as follows.
Algorithm A . Choose an initial point , a symmetric matrix , let , , compute , let .
Step 1. Compute .
Step 2. If , stop.
Step 3. Solve the subproblems (5) and (6) to get the quasinormal step and the tangential step . Let .
Step 4. If , let , then go to Step 8.
Step 5. If is not acceptable to the filter, go to Step 8. Otherwise and update the filter according to (21) and (22), the trust region radius and , then get the corresponding .
Step 6. If , and go to Step 1, otherwise go to Step 7.
Step 7. By restoration Algorithm B to get , then the trial point .
Step 8. Update the trust region radius by Algorithm C, let and go to Step 3.
We aim to reduce the value of in the restoration Algorithm B, that is to get by Newtontype method.
Algorithm B . Let .
Step 1. If and is acceptable by the filter, then , stop.
Step 2. Compute to get , then compute .
Step 3. If , and go to Step 2.
Step 4. If , , compute and go to Step 1, where .
Algorithm C (updating the trust region radius). Given , we have the following.(1)If or is not acceptable to the filter, .(2)If is acceptable to the filter but does not satisfiy condition (27), .(3)If is acceptable to the filter and satisfies (27), .
From the description above and the idea of the algorithm, we can see that our algorithm is more flexible. Every successful iterate must be any better than the predecessor one in some degree according to the traditional filter method. But our algorithm relaxes this demand by using the nonmonotone technique and also avoids Maratos effect in a certain degree. Moreover, Algorithm C allows a relatively wide choice of the trust region.
5. The Convergence Properties
In this section, to present a proof of global convergence of algorithm, we always assume that the following conditions hold.
Assumption
(A1) The objective function and the constraint functions are twice continuously differentiable. (A2) For all , , and all remain in a closed, bounded convex subset . (A3) The matrix is nonsingular matrix for all . (A4) The matrices are uniformly bounded in , where denotes a matrix whose columns form a basis of the null space of . (A5) The matrix is uniformly bounded.
By the assumptions, we can suppose there exist constants such that .
By (A1) and (A2), it holds where , hence in the plane, the pair lies in the area .
From (A1), (A2), and (A3), it exists a constant such that
Lemma 2. At the current iterate , let the trial point component actually be normal to the tangential space. Under the problem assumptions, there exists a constant independent of the iterates such that
Proof. It is similar to the proof of Lemma 2 in [13].
Lemma 3. Under Assumptions, there exist positive constants independent of the iterates such that
Proof. The proof is an application of Lemma 1 to the two subproblems (5) and (6).
Lemma 4. Suppose that Assumptions hold, then restoration Algorithm B is well defined.
Proof. The conclusion is obvious, if . Otherwise it exists such that for all , it holds . Consider the set
where . By Lemma 3 and the definition of , we have
By , it holds for . From Algorithm B, we can obtain that for all .
On the other side,
for . By the algorithm, the radius should be satisfied , that is contradicted to . The proof is complete.
Now, we analyze the impact of the criteria (23) and (24). Once a trial point is accepted as a new iterate, it must be provided some improvement, and we formalize this by saying that iterate improves on iterate . That is the trial point is accepted at iterate ; it happens under two situations, one is by the criteria (23), that is, the other is by the criteria (24), that is,
Now consider any iterate , it improved on , which was itself accepted because it improved on , and so on, until back to . Hence we may construct a chain of successful iterations indexed by = for each , such that where is the smallest index in the chain of successful iterations.
Lemma 5. Suppose that Assumptions hold and Algorithm A does not terminate finitely, apply Algorithm A to the problem , then for all and , it holds
Proof. For , if , by (24), , then
If ,
If ,
If , . By , it holds
Then from (23), . It implies (43). Moreover
Together with (42) and (43), the result follows.
Lemma 6. Suppose that Assumptions hold. If Algorithm A does not terminate finitely and the filter contains infinite iterates, then .
Proof. Suppose by contradiction that there exists a constant and infinite sequence such that for all . Because there are infinite iterations in the filter, we have , then for . Then by (31), is upper bounded for each . That means it exists such that , so . Hence must be finite, it contradicts to the infinity of . The proof is complete.
Lemma 7. Suppose that Assumptions hold and Algorithm A terminate finitely, then .
Proof. From the Algorithm A and the definition of filter, the conclusion follows.
Lemma 8. For any trial point , there must be one accepted by the filter.
Lemma 9. Suppose that Assumptions hold, there exists independent of the iterates such that
Proof. By (32), the assumptions and , it is obvious that The proof is complete.
Lemma 10. Suppose that Assumptions hold and , if one can deduce
Proof. By the assumptions and the definition of , it holds From Lemma 3, . Together with then . It is the conclusion.
Lemma 11. Suppose the conditions of Lemma 10 hold, if then .
Proof. From the definition of and Lemma 10, together with (31), we have It is obvious that .
Lemma 12. Suppose the conditions of Lemmas 10 and 11 hold, if then
Proof. By Lemmas 3, 10, and 11, together with , it holds From the Algorithm, , then Hence .
Theorem 13. Suppose the assumptions hold, there must exist such that for each , it holds
Proof. Let be large enough such that , it is true by Lemmas 6 and 7. Suppose by contradiction that the index is the first one after , which satisfies
where is the smallest value of violation function in filter. Then . By the above analysis, we know , that is . From the Algorithm and (60), it concludes
By (60) and (61), (53) can be obtained. In Lemma 11, let instead of , it deduces
Based on Lemma 12, together with (60), (61), and the algorithm, we can see
It can be seen that (53) is true for , with (55), we can deduce
That means can be accepted by the filter. From above and (55), we know . Hence the index is not the first one after which satisfied (60), that is a contradiction. So, for any , it holds . Define
we can see that
holds for each . The proof is complete.
Lemma 14. Suppose that Assumptions hold and Algorithm A does not terminate finitely, then .
Proof. Suppose by contradiction that for , there exists a constant such that .
By Assumption (A3) and (A4), . From Lemma 6, we know . Hence there exists such that
for . Then for .
It is obvious that
By the proof of Lemma 9, it holds . Together with Lemma 6 and the definition of , we have
By the Algorithm, we can get
Then
By (68), (69), and (71), it deduces
Based on the assumptions, Lemma 3 and Theorem 13, for , it holds
which contradicts (72). The conclusion follows.
Theorem 15. Suppose the assumptions hold, and apply the algorithm to problem , then where , denotes a matrix whose columns form a basis of the null space of .
Proof. If the algorithm terminates finitely, it is obvious that it holds. Otherwise, by Lemmas 6 and 14, the conclusion also can be obtained.
Theorem 16. Suppose the assumptions hold, and is the infinite sequence obtained by the algorithm, then there must exist a subsequence such that and satisfies the one order KKT condition of .
Proof. By Assumption (A1), there exist a subsequence and , such that . Together with Assumption (A3) and (A4), it hods , which means for large enough , lies in the space spaned by the columns of . That is there exists such that The conclusion follows.
6. Some Numerical Experiments
(1)Updating of is done by , where and , is the multipluser of corresponding quadratic subproblems.(2)We assume the error toleration is .(3)The algorithm parameters were set as follows: , , , , , , , , , , , , . The program is written in Matlab.
The numerical results for the test problems are listed in Table 1.

In Table 1, the problems are numbered in the same way as in Schittkowski [20] and Hock and Schittkowski [21]. For example, “S216” is the problem in Schittkowski [20] and “HS6” is the problem in Hock and Schittkowski [21]. NF, NG represent the numbers of function and gradient calculations and “L's” is the solution in [22]. The numerical results show that the our algorithm is more effective than the L's for most test examples. Moreover, the higher the level of nonmonotonic, the better the numerical results. The results show that the new algorithm is robust and effective, and more flexible for the acceptance of the the trial iterate.
Acknowledgments
This research is supported by the National Natural Science Foundation of China (no. 11101115) and the Natural Science Foundation of Hebei Province (nos. A2010000191, Y2012021).
References
 R. H. Byrd, R. B. Schnabel, and G. A. Shultz, “A trust region algorithm for nonlinearly constrained optimization,” SIAM Journal on Numerical Analysis, vol. 24, no. 5, pp. 1152–1170, 1987. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 J. E. Dennis Jr., M. ElAlem, and M. C. Maciel, “A global convergence theory for general trustregionbased algorithms for equality constrained optimization,” SIAM Journal on Optimization, vol. 7, no. 1, pp. 177–207, 1997. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 M. J. D. Powell and Y. Yuan, “A trust region algorithm for equality constrained optimization,” Mathematical Programming A, vol. 49, no. 2, pp. 189–211, 1991. View at: Publisher Site  Google Scholar  MathSciNet
 R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Mathematical Programming A, vol. 91, no. 2, pp. 239–269, 2002. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 C. M. Chin and R. Fletcher, “On the global convergence of an SLPfilter algorithm that takes EQP steps,” Mathematical Programming A, vol. 96, no. 1, pp. 161–177, 2003. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A. Wächter, “Global convergence of a trustregion SQPfilter algorithm for general nonlinear programming,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 635–659, 2002. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 R. Fletcher, S. Leyffer, and P. L. Toint, “On the global convergence of a filterSQP algorithm,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 44–59, 2002. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 S. Ulbrich, “On the superlinear local convergence of a filterSQP method,” Mathematical Programming B, vol. 100, no. 1, pp. 217–245, 2004. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 M. Ulbrich, S. Ulbrich, and L. N. Vicente, “A globally convergent primaldual interiorpoint filter method for nonlinear programming,” Mathematical Programming A, vol. 100, no. 2, pp. 379–410, 2004. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 R. Fletcher and S. Leyffer, “A bundle filter method for nonsmooth nonlinear optimization,” Tech. Rep. NA/195, Department of Mathematics, University of Dundee, Dundee, Scotland, December 1999. View at: Google Scholar
 L. Grippo, F. Lampariello, and S. Lucidi, “A nonmonotone line search technique for Newton's method,” SIAM Journal on Numerical Analysis, vol. 23, no. 4, pp. 707–716, 1986. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 M. Ulbrich and S. Ulbrich, “Nonmonotone trust region methods for nonlinear equality constrained optimization without a penalty function,” Mathematical Programming B, vol. 95, no. 1, pp. 103–135, 2003. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 K. Su and D. Pu, “A nonmonotone filter trust region method for nonlinear constrained optimization,” Journal of Computational and Applied Mathematics, vol. 223, no. 1, pp. 230–239, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 K. Su and Z. Yu, “A modified SQP method with nonmonotone technique and its global convergence,” Computers & Mathematics with Applications, vol. 57, no. 2, pp. 240–247, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 N. I. M. Gould and P. L. Toint, “Global convergence of a nonmonotone trustregion filter algorithm for nonlinear programming,” in Multiscale optimization methods and applications, vol. 82, pp. 125–150, Springer, New York, NY, USA, 2006. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Z. Chen and X. Zhang, “A nonmonotone trustregion algorithm with nonmonotone penalty parameters for constrained optimization,” Journal of Computational and Applied Mathematics, vol. 172, no. 1, pp. 7–39, 2004. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Z. W. Chen, “A penaltyfreetype nonmonotone trustregion method for nonlinear constrained optimization,” Applied Mathematics and Computation, vol. 173, no. 2, pp. 1014–1046, 2006. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. A. M. Gomes, M. C. Maciel, and J. M. Martínez, “Nonlinear programming algorithms using trust regions and augmented Lagrangians with nonmonotone penalty parameters,” Mathematical Programming A, vol. 84, no. 1, pp. 161–200, 1999. View at: Google Scholar  Zentralblatt MATH  MathSciNet
 M. J. D. Powell, “Convergence properties of a class of minimization algorithms,” in Nonlinear Programming, pp. 1–27, Academic Press, New York, NY, USA, 1974. View at: Google Scholar  MathSciNet
 K. Schittkowski, More Test Examples for Nonlinear Programming Codes, vol. 282, Springer, Berlin, Germany, 1987. View at: Publisher Site  MathSciNet
 W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, vol. 187, Springer, Berlin, Germany, 1981. View at: Publisher Site  MathSciNet
 M. Lalee, J. Nocedal, and T. Plantenga, “On the implementation of an algorithm for largescale equality constrained optimization,” SIAM Journal on Optimization, vol. 8, no. 3, pp. 682–706, 1998. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
Copyright
Copyright © 2013 Ke Su et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.