Operational ResearchView this Special Issue
Research Article | Open Access
Ke Su, Wei Liu, Xiaoli Lu, "Global Convergence of a New Nonmonotone Filter Method for Equality Constrained Optimization", Journal of Applied Mathematics, vol. 2013, Article ID 976509, 9 pages, 2013. https://doi.org/10.1155/2013/976509
Global Convergence of a New Nonmonotone Filter Method for Equality Constrained Optimization
A new nonmonotone filter trust region method is introduced for solving optimization problems with equality constraints. This method directly uses the dominated area of the filter as an acceptability criterion for trial points and allows the dominated area decreasing nonmonotonically. Compared with the filter-type method, our method has more flexible criteria and can avoid Maratos effect in a certain degree. Under reasonable assumptions, we prove that the given algorithm is globally convergent to a first order stationary point for all possible choices of the starting point. Numerical tests are presented to show the effectiveness of the proposed algorithm.
We analyze an algorithm for solving optimization problems where a smooth objective function is to be minimized subject to smooth nonlinear equality constraints. More formally, we consider the problem, where , the functions and are all twice continuously differentiable. For convenience, let and , and refers to , to , to and to , and so forth.
There are many trust region methods for equality constrained nonlinear programming , for example, Byrd et al. , Dennis Jr. et al.  and Powell and Yuan , but in these works, a penalty or augmented Lagrange function is always used to test the acceptability of the iterates. However, there are several difficulties associated with the use of penalty function, and in particular the choice of the penalty parameter. Hence, in 2002, Fletcher and Leyffer  proposed a class of filter method, which does not require any penalty parameter and has promising numerical results. Consequently, filter technique has been employed to many approaches, for instance, SLP methods , SQP methods [6–8], interior point approaches , bundle techniques , and so on.
Filter technique, in fact, exhibits a certain degree of nonmonotonicity. The nonmonotone technique was proposed by Grippo et al. in 1986  and combined with many other methods. M. Ulbrich and S. Ulbrich  proposed a class of penalty-function-free nonmonotone trust region methods for nonlinear equality constrained optimization without filter technique. Su and Pu  introduced a nonmonotone trust region method which used the nonmonotone technique in the traditional filter criteria. Su and Yu  presented a nonmonotone method without penalty function or filter. Gould and Toint  directly used the dominated area of the filter as an acceptability criteria for trial points and obtained the global convergence properties. We refer the reader [16–18] for some works about this issue.
Motivated by the ideas and methods above, we propose a modified nonmonotone filter trust region method for solving problem . Similar to the Byrd-Omojokun class of algorithms, each step is decomposed into the sum of two distinct components, a quasi-normal step and a tangential step. The main contribution of our paper is to employ the nonmonotone idea to the dominated area of the filter so that the new and more flexible criteria is given, which is different from that of Gould and Toint  and Su and Pu . Under usual assumptions, we prove that the given algorithm is globally convergent to first order stationary points.
This paper is organized as follows. In Section 2, we introduce the fraction of Cauchy decrease and the composite SQP step. The new nonmonotone filter technique is given in Section 3. In Section 4, we propose the new nonmonotone filter method and present the global convergence properties in Section 5. Some numerical results are reported in the last section.
2. The Fraction of Cauchy Decrease and the Composite SQP Step
Consider the following unconstraint minimization optimization problem: where is a continuously differentiable function. A trust region algorithm for solving the above problem is an iterate procedure that computes a trial step as an approximate solution to the following subproblems: where is the Hessian matrix or an approximate to it and is a given trust region radius.
To assure the global convergence, the step is required only to satisfy a fraction of Cauchy decrease condition. This means that must predict via the quadratic model function at least as much as a fraction of the decreased given by the Cauchy step on ; that is, there exists a constant fixed across all iterations, such that where is the steepest descent step for inside the trust region.
Lemma 1. If the trial step satisfies a fraction of Cauchy decrease condition, then
Proof (see Powell  for the proof). Now, we turn to explain the composite SQP step. Given an approximate estimate of the solution at th iteration, following Dennis Jr. et al.  and M. Ulbrich and S. Ulbrich , we obtain the trial step by computing a quasi-normal step and a tangential step . The purpose of the quasi-normal step is to improve feasibility. To improve optimality, we seek in the tangential space of the linearized constraints in such a way that it provides sufficient decrease for a quadratic model of the objective function . Let , where is a symmetric approximation of .
is the solution to the subproblem where is a trust region radius and . In order to improve the value of the objective function, we solve the following subproblem to get
Then we get the current trial step . Let , where and denote a matrix whose columns form a basis of the null space of . We refer to  for a more detailed discussion of this issue.
In usual way that impose a trust region in step-decomposition methods, the quasi-normal step and the tangential step are required to satisfy where . Here, to simplify the proof, we only impose a trust region on and , which is natural.
Note that is the reduced gradient of in terms of the representation of the tangential step:
Then the first order necessary optimality conditions (Karush-Kuhn-Tucker or KKT conditions) at a local solution of problem can be written as
3. A New Nonmonotone Filter Technique
In filter method, originally proposed by Fletcher and Leyffer , the acceptability of iterates is determined by comparing the value of constraint violation and the objective function with previous iterates collected in a filter. Define the violation function by , it is easy to see that if and only if is a feasible point, so a trial point should reduce either the value of constraint violation or that of the objective function .
In the process of the algorithm, we need to decide whether the trial point is any better than as an approximate solution to the problem . If we decide that this is the case, we say that the iteration is successful and choose as the next iterate. Let us denote by the set of all successful iterations, that is,
In traditional filter method, a point is called acceptable to the filter if and only if where , denotes the filter set. Define
A trial point is accepted if and only if .
Now, similar to the idea of Gould and Toint , we give a new modified nonmonotone filter technique. For any -pair, define an area that represents its contribution to the area of , we hope this contribution is positive; that is, the area of is increasing. For convenience, we partition the right half-plane into four different regions (see Figure 1). Define to be the complement of . Let
These four parts are(1)the dominated part of the filter: ;(2)the undominated part of lower left corner of the half plane: (3)the undominated upper left corner: ;(4)the undominated lower right corner: .
Consider the trial point , if the filter is empty, then define its contribution to the area of the filter by where is a constant. If the filter is not empty, then define the contribution of to the area of the filter by four different formulae.
If , assume
If , assume
If , assume
If , assume where , and .
Figure 2 illustrate the corresponding areas in the filter. Horizontally dashed surfaces indicate a positive contribution and vertically dashed ones a negative contribution. Note that is a continuous function of .
Next, we should consider the updating of the filter. If , then
If , then
We now return to the question of deciding whether a trial point is acceptable for the filter or not. We will insist that this is a necessary condition for the iteration to be successful in the sense that . If we consider an iterate , there must exist a predecessor iteration such that . Under the monotonic situation, a trial point would be accepted whenever it results in an sufficient increase in the dominated area of the filter, that means would be accepted whenever where , is a constant. Under the nonmonotonic situation, we relax condition (23) to be where , is some reference iteration for , , , .
4. The New Nonmonotone Filter Trust Region Algorithm
Our algorithm is based on the usual trust region technique; define the predict reduction for the function to be and the actual reduction
Moreover, let , if there exists a nonzero constant such that and condition (23) and (24) hold, the trial point will be called acceptable. Then the next trial point is obtained, and for its feasibility, we consider the condition is true or not, where and are all positive constants, if it is not true, then turn to the feasibility restoration phase and define A formal description of the algorithm is given as follows.
Algorithm A . Choose an initial point , a symmetric matrix , let , , compute , let .
Step 1. Compute .
Step 2. If , stop.
Step 4. If , let , then go to Step 8.
Step 7. By restoration Algorithm B to get , then the trial point .
Step 8. Update the trust region radius by Algorithm C, let and go to Step 3.
We aim to reduce the value of in the restoration Algorithm B, that is to get by Newton-type method.
Algorithm B . Let .
Step 1. If and is acceptable by the filter, then , stop.
Step 2. Compute to get , then compute .
Step 3. If , and go to Step 2.
Step 4. If , , compute and go to Step 1, where .
Algorithm C (updating the trust region radius). Given , we have the following.(1)If or is not acceptable to the filter, .(2)If is acceptable to the filter but does not satisfiy condition (27), .(3)If is acceptable to the filter and satisfies (27), .
From the description above and the idea of the algorithm, we can see that our algorithm is more flexible. Every successful iterate must be any better than the predecessor one in some degree according to the traditional filter method. But our algorithm relaxes this demand by using the nonmonotone technique and also avoids Maratos effect in a certain degree. Moreover, Algorithm C allows a relatively wide choice of the trust region.
5. The Convergence Properties
In this section, to present a proof of global convergence of algorithm, we always assume that the following conditions hold.
(A1) The objective function and the constraint functions are twice continuously differentiable. (A2) For all , , and all remain in a closed, bounded convex subset . (A3) The matrix is nonsingular matrix for all . (A4) The matrices are uniformly bounded in , where denotes a matrix whose columns form a basis of the null space of . (A5) The matrix is uniformly bounded.
By the assumptions, we can suppose there exist constants such that .
By (A1) and (A2), it holds where , hence in the -plane, the -pair lies in the area .
From (A1), (A2), and (A3), it exists a constant such that
Lemma 2. At the current iterate , let the trial point component actually be normal to the tangential space. Under the problem assumptions, there exists a constant independent of the iterates such that
Proof. It is similar to the proof of Lemma 2 in .
Lemma 3. Under Assumptions, there exist positive constants independent of the iterates such that
Lemma 4. Suppose that Assumptions hold, then restoration Algorithm B is well defined.
Proof. The conclusion is obvious, if . Otherwise it exists such that for all , it holds . Consider the set
where . By Lemma 3 and the definition of , we have
By , it holds for . From Algorithm B, we can obtain that for all .
On the other side, for . By the algorithm, the radius should be satisfied , that is contradicted to . The proof is complete.
Now, we analyze the impact of the criteria (23) and (24). Once a trial point is accepted as a new iterate, it must be provided some improvement, and we formalize this by saying that iterate improves on iterate . That is the trial point is accepted at iterate ; it happens under two situations, one is by the criteria (23), that is, the other is by the criteria (24), that is,
Now consider any iterate , it improved on , which was itself accepted because it improved on , and so on, until back to . Hence we may construct a chain of successful iterations indexed by = for each , such that where is the smallest index in the chain of successful iterations.
Lemma 6. Suppose that Assumptions hold. If Algorithm A does not terminate finitely and the filter contains infinite iterates, then .
Proof. Suppose by contradiction that there exists a constant and infinite sequence such that for all . Because there are infinite iterations in the filter, we have , then for . Then by (31), is upper bounded for each . That means it exists such that , so . Hence must be finite, it contradicts to the infinity of . The proof is complete.
Lemma 7. Suppose that Assumptions hold and Algorithm A terminate finitely, then .
Proof. From the Algorithm A and the definition of filter, the conclusion follows.
Lemma 8. For any trial point , there must be one accepted by the filter.
Lemma 9. Suppose that Assumptions hold, there exists independent of the iterates such that
Proof. By (32), the assumptions and , it is obvious that The proof is complete.
Lemma 10. Suppose that Assumptions hold and , if one can deduce
Proof. By the assumptions and the definition of , it holds From Lemma 3, . Together with then . It is the conclusion.
Lemma 11. Suppose the conditions of Lemma 10 hold, if then .
Theorem 13. Suppose the assumptions hold, there must exist such that for each , it holds
Proof. Let be large enough such that , it is true by Lemmas 6 and 7. Suppose by contradiction that the index is the first one after , which satisfies
where is the smallest value of violation function in filter. Then . By the above analysis, we know , that is . From the Algorithm and (60), it concludes
By (60) and (61), (53) can be obtained. In Lemma 11, let instead of , it deduces
Based on Lemma 12, together with (60), (61), and the algorithm, we can see
It can be seen that (53) is true for , with (55), we can deduce
That means can be accepted by the filter. From above and (55), we know . Hence the index is not the first one after which satisfied (60), that is a contradiction. So, for any , it holds . Define
we can see that holds for each . The proof is complete.
Lemma 14. Suppose that Assumptions hold and Algorithm A does not terminate finitely, then .
Proof. Suppose by contradiction that for , there exists a constant such that .
By Assumption (A3) and (A4), . From Lemma 6, we know . Hence there exists such that for . Then for .
It is obvious that
By the proof of Lemma 9, it holds . Together with Lemma 6 and the definition of , we have By the Algorithm, we can get Then By (68), (69), and (71), it deduces Based on the assumptions, Lemma 3 and Theorem 13, for , it holds which contradicts (72). The conclusion follows.
Proof. By Assumption (A1), there exist a subsequence and , such that . Together with Assumption (A3) and (A4), it hods , which means for large enough , lies in the space spaned by the columns of . That is there exists such that The conclusion follows.
6. Some Numerical Experiments
(1)Updating of is done by , where and , is the multipluser of corresponding quadratic subproblems.(2)We assume the error toleration is .(3)The algorithm parameters were set as follows: , , , , , , , , , , , , . The program is written in Matlab.
The numerical results for the test problems are listed in Table 1.
In Table 1, the problems are numbered in the same way as in Schittkowski  and Hock and Schittkowski . For example, “S216” is the problem in Schittkowski  and “HS6” is the problem in Hock and Schittkowski . NF, NG represent the numbers of function and gradient calculations and “L's” is the solution in . The numerical results show that the our algorithm is more effective than the L's for most test examples. Moreover, the higher the level of nonmonotonic, the better the numerical results. The results show that the new algorithm is robust and effective, and more flexible for the acceptance of the the trial iterate.
This research is supported by the National Natural Science Foundation of China (no. 11101115) and the Natural Science Foundation of Hebei Province (nos. A2010000191, Y2012021).
- R. H. Byrd, R. B. Schnabel, and G. A. Shultz, “A trust region algorithm for nonlinearly constrained optimization,” SIAM Journal on Numerical Analysis, vol. 24, no. 5, pp. 1152–1170, 1987.
- J. E. Dennis Jr., M. El-Alem, and M. C. Maciel, “A global convergence theory for general trust-region-based algorithms for equality constrained optimization,” SIAM Journal on Optimization, vol. 7, no. 1, pp. 177–207, 1997.
- M. J. D. Powell and Y. Yuan, “A trust region algorithm for equality constrained optimization,” Mathematical Programming A, vol. 49, no. 2, pp. 189–211, 1991.
- R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Mathematical Programming A, vol. 91, no. 2, pp. 239–269, 2002.
- C. M. Chin and R. Fletcher, “On the global convergence of an SLP-filter algorithm that takes EQP steps,” Mathematical Programming A, vol. 96, no. 1, pp. 161–177, 2003.
- R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A. Wächter, “Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 635–659, 2002.
- R. Fletcher, S. Leyffer, and P. L. Toint, “On the global convergence of a filter-SQP algorithm,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 44–59, 2002.
- S. Ulbrich, “On the superlinear local convergence of a filter-SQP method,” Mathematical Programming B, vol. 100, no. 1, pp. 217–245, 2004.
- M. Ulbrich, S. Ulbrich, and L. N. Vicente, “A globally convergent primal-dual interior-point filter method for nonlinear programming,” Mathematical Programming A, vol. 100, no. 2, pp. 379–410, 2004.
- R. Fletcher and S. Leyffer, “A bundle filter method for nonsmooth nonlinear optimization,” Tech. Rep. NA/195, Department of Mathematics, University of Dundee, Dundee, Scotland, December 1999.
- L. Grippo, F. Lampariello, and S. Lucidi, “A nonmonotone line search technique for Newton's method,” SIAM Journal on Numerical Analysis, vol. 23, no. 4, pp. 707–716, 1986.
- M. Ulbrich and S. Ulbrich, “Non-monotone trust region methods for nonlinear equality constrained optimization without a penalty function,” Mathematical Programming B, vol. 95, no. 1, pp. 103–135, 2003.
- K. Su and D. Pu, “A nonmonotone filter trust region method for nonlinear constrained optimization,” Journal of Computational and Applied Mathematics, vol. 223, no. 1, pp. 230–239, 2009.
- K. Su and Z. Yu, “A modified SQP method with nonmonotone technique and its global convergence,” Computers & Mathematics with Applications, vol. 57, no. 2, pp. 240–247, 2009.
- N. I. M. Gould and P. L. Toint, “Global convergence of a non-monotone trust-region filter algorithm for nonlinear programming,” in Multiscale optimization methods and applications, vol. 82, pp. 125–150, Springer, New York, NY, USA, 2006.
- Z. Chen and X. Zhang, “A nonmonotone trust-region algorithm with nonmonotone penalty parameters for constrained optimization,” Journal of Computational and Applied Mathematics, vol. 172, no. 1, pp. 7–39, 2004.
- Z. W. Chen, “A penalty-free-type nonmonotone trust-region method for nonlinear constrained optimization,” Applied Mathematics and Computation, vol. 173, no. 2, pp. 1014–1046, 2006.
- F. A. M. Gomes, M. C. Maciel, and J. M. Martínez, “Nonlinear programming algorithms using trust regions and augmented Lagrangians with nonmonotone penalty parameters,” Mathematical Programming A, vol. 84, no. 1, pp. 161–200, 1999.
- M. J. D. Powell, “Convergence properties of a class of minimization algorithms,” in Nonlinear Programming, pp. 1–27, Academic Press, New York, NY, USA, 1974.
- K. Schittkowski, More Test Examples for Nonlinear Programming Codes, vol. 282, Springer, Berlin, Germany, 1987.
- W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, vol. 187, Springer, Berlin, Germany, 1981.
- M. Lalee, J. Nocedal, and T. Plantenga, “On the implementation of an algorithm for large-scale equality constrained optimization,” SIAM Journal on Optimization, vol. 8, no. 3, pp. 682–706, 1998.
Copyright © 2013 Ke Su et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.