Research Article | Open Access
Zhong Jin, Yuqing Wang, "An Improved Line Search Filter Method for the System of Nonlinear Equations", Journal of Applied Mathematics, vol. 2012, Article ID 273141, 15 pages, 2012. https://doi.org/10.1155/2012/273141
An Improved Line Search Filter Method for the System of Nonlinear Equations
An improved line search filter algorithm for the system of nonlinear equations is presented. We divide the equations into two groups, one contains the equations that are treated as equality constraints and the square of other equations is regarded as objective function. Two groups of equations are updated at every iteration in the works by Nie (2004, 2006, and 2006), by Nie et al. (2008), and by Gu (2011), while we just update them at the iterations when it is needed indeed. As a consequence, the scale of the calculation is decreased in a certain degree. Under some suitable conditions the global convergence can be induced. In the end, numerical experiments show that the method in this paper is effective.
Many applied problems are reduced to solve the system of nonlinear equations, which is one of the most basic problems in mathematics. This task has applications in many scientific fields such as physics, chemistry, and economics. More formally, the problem to be solved is stated as follows where each is a smooth function.
A well-known method for solving nonlinear equations is the Newton method, an iterative scheme which is locally quadratical convergent only if the initial iteration is sufficiently close to the solution. To improve the global properties, some important algorithms  for nonlinear equations proceed by minimizing a least square problem: which can be also handled by the Newton method, while Powell  gives a counterexample to show a dissatisfactory fact that the iterates generated by the above least square problem may converge to a nonstationary point of .
Traditionally, a penalty or augmented Lagrange function is always used as a merit function to test the acceptability of the iterates. However, as we all know, there are several difficulties associated with the use of penalty function, and in particular the choice of the penalty parameter. Too low a choice may result in an optimal solution that cannot be obtained, on the other hand, too large a choice damps out the effect of the objective function. Hence, filter method has been first introduced for constrained nonlinear optimization problems in a sequential quadratic programming (SQP) trust-region algorithm by Fletcher and Leyffer , offering an alternative to merit functions, as a tool to guarantee global convergence in algorithms for nonlinear programming (NLP). The computational results are very promising in . Furthermore, Fletcher et al. [4, 5] give the global convergence of the trust-region filter-SQP method, then Ulbrich  gets its superlinear local convergence. Consequently, filter method has been actually applied in many optimization techniques [7–13], for instance the pattern search method , the SLP method , the interior method , the bundle approaches [10, 11], and so on. Also combined with the trust-region search technique, Gould et al. extended the filter method to the system of nonlinear equations (1.1) and nonlinear least squares in , and to the unconstrained optimization problem with multidimensional filter technique in . In addition, Wächter and Biegler [16, 17] presented line search filter methods for nonlinear equality constrained programming and the global and local convergence were given.
Recently, some other ways were given to attack the problem (1.1) (see [18–22]). There are two common features in these papers, one is the filter approach that is utilized, and the other is that at every iteration the system of nonlinear equations is transformed into a constrained nonlinear programming problem and the equations are divided into two groups, some equations are treated as constraints and the others act as the objective function. For instance combined with the filter line search technique [16, 17], the system of nonlinear equations in  at the kth iteration becomes the following optimization problem with equality constraints: where the sets and are defined as and for some positive constant such that .
Motivated by the ideas and methods above, we propose an improved line search filter method for the system of nonlinear equations. We also divide the equations into two groups, one contains the equations that are treated as equality constraints and the square of other equations is regarded as objective function. Two groups of equations are updated at every iteration in those works [18–22], while we just update them at the iterations when it is needed indeed. Specifically, using similar transformed optimization problem (1.3), the difference between our method with  is that the sets and in (1.3) need not be updated at every iteration. As a consequence, the scale of the calculation is decreased in a certain degree. In our algorithm two groups of equations cannot be changed after a f-type iteration, thus in the case that , the two groups are fixed after finite number of iterations. In addition, the filter should not be updated after a f-type iteration, so naturally the global convergence is discussed, respectively, according to whether the number of the updated filter is infinite or not. And it is shown that every limit point of the sequence of iterates generated by the algorithm is the solution to (1.1) or a local infeasible point when . Furthermore, the global convergent property is induced under mild assumptions. In the end, numerical experiments show that the method in this paper is effective.
The paper is outlined as follows. In Section 2, the line search filter method is developed and its key ingredients are described. In Section 3 we prove that under suitable conditions, the method is well defined and globally convergent. Finally, some numerical results are given in Section 4.
2. Description of the Algorithm
To solve the system of nonlinear Equation (1.1), we also transform it into the optimization problem (1.3) where equations are divided into two groups as the one in , then let and . The linearization of the KKT condition of (1.3) at the kth iteration is as follows: where is the Hessian or approximate Hessian matrix of , and .
After a search direction has been computed, a step size is determined in order to obtain the trial iteration More precisely, for fixed constants , we say that a trial step size provides sufficient reduction with respect to the current if
For the sake of a simplified notation, we define the filter in this paper not as a list but as a set containing all -pairs which are prohibited in iteration . We say that a trial point is acceptable to the filter if its -pair does not lie in the taboo region, that is, if At the beginning of the optimization, the filter is initialized to be empty: . Throughout the optimization the filter is then augmented in some iterations after the new iterate has been accepted. For this, the following updating formula is used:
Similar to the traditional strategy of the filter method, to avoid obtaining a feasible point but not an optimal solution, we consider the following f-type switching condition: where , and .
When Condition (2.6) holds, the step is a descent direction for current objective function. Then, instead of insisting on (2.3), the Armijo-type reduction condition is employed as follows: where is a fixed constant.
If (2.6) and (2.7) hold for the accepted trial step size, we may call it an f-type point, and accordingly this iteration is called an f-type iteration. An f-type point should be accepted as with no updating of the filter, that is, While if a trial point does not satisfy the switching condition (2.6) but satisfies (2.3), we call it an h-type point (or accordingly an h-type iteration). An h-type point should be accepted as with updating of the filter.
In the situation, where no admissible step size can be found, the method switches to a feasibility restoration stage, whose purpose is to find a new iterate that satisfies (2.3) and is also acceptable to the current filter by trying to decrease the constraint violation. In order to detect the situation where no admissible step size can be found and the restoration phase has to be invoked, we define that
We are now ready to formally state the overall algorithm for solving the the system of nonlinear Equation (1.1).
Algorithm 2.1. We have the following steps.
Step 1. Initialization: Choose an initial point , and . Compute , , , and for . Set and .
Step 2. If then stop. Otherwise compute (2.1) to obtain . If there exists no solution to (2.1), go to Step 6. If then stop.
Step 3. Let and . Compute by (2.9).
Step 3.1. If , go to Step 6. Otherwise compute . If , go to Step 3.2. Step . Case 1. The switching condition (2.6) holds. If the reduction condition (2.7) holds, set , , , and go to Step 5. Otherwise, go to Step 3.2. Step . Case 2. The switching condition (2.6) is not satisfied. If (2.3) holds, set , augment the filter using (2.5) and go to Step 4. Otherwise, go to Step 3.2.
Step 3.2. Choose . Set and go to Step 3.1.
Step 4. Compute and by (1.3). If , set and .
Step 5. Compute , and . Go to Step 2 with replaced by .
Step 6. (Feasibility Restoration Stage) Find such that is accepted by the filter and the infeasibility is reduced. Go to Step 2.
3. Global Convergence of Algorithm
In the reminder of this paper we denote the set of indices of those iterations in which the filter has been augmented by . Let us now state the assumptions necessary for the global convergence analysis.
Assumption 3.1. The sequence generated by Algorithm 2.1 is nonempty and bounded.
Assumption 3.2. The functions , are all twice continuously differentiable on an open set containing .
Assumption 3.3. There exist two constants such that the matrices sequence satisfy for all and .
Assumption 3.4. has full column rank and for all with a positive constant .
By Assumption 3.1 we have . Since and , then associated with the filter is restricted to
Proof. The proof of this theorem can be found in .
Next we can assume that our algorithm does not terminate finitely and an infinite sequence of points is generated.
Lemma 3.6. Under the above assumptions, there exists a solution to (2.1) with exact (or inexact) line search which satisfies the following descent conditions, where , , and are all positive constants independent of .
Proof. By virtue of Taylor expansion of with , we obtain
where the last inequality follows the above assumptions and . Also, from (2.1) we immediately get that , that is, . With , thereby,
then (3.2) consequently holds.
According to Taylor expansion of (i.e., ), we then have where the last inequality follows the first two assumptions and . That is to say, which is just (3.3).
Theorem 3.7. If there are only finite or infinite number of iterates entering the filter, then
Proof. Consider the following.
Case 1 (if ). The proof is by contraction. Suppose that there exists an infinite subsequence of such that for some . At each iteration , is added to the filter which means that no other can be added to the filter at a later stage within the area and the area of each of these squares is at least .
Thus the is completely covered by at most a finite number of such areas in contraction to the infinite subsequence satisfying (3.9). This means that (3.8) is true.
Case 2 (if ). From , we know the filter updates in a finite number, then there exists , for the filter does not update. As h-type iteration and restoration algorithm all need the updating of the filter, then for our algorithm only executes the f-type iterations.
Because and do not change in the f-type iteration, then for we have and . Thus we obtain
From the switching condition (2.6) we know that together with the reduction condition (2.7) we get and since for , this reduces to
Suppose to the contrary that there exists an infinite subsequence of such that for some . Choose an arbitrary subsequence of satisfying , we find that which is a contraction.
Lemma 3.8. Let be a subsequence of iterates for which (2.6) hold and have the same and . Then there exists a such that
Lemma 3.9. If there exists such that for it holds , where , then there exist and a constant , for such that
Proof. From (2.1) we obtain and , so As implies , then there exists , for such that . Thus with .
Theorem 3.10. Suppose that is an infinite sequence generated by Algorithm 2.1 and , one has Namely, every limit point is the solution to (1.1) or a local infeasible point. If the gradients of are linear independent for all and , then the solution to (1.1) is obtained.
Proof. As was noted in the proof of the case 2 in Theorem 3.7, from , the algorithm only executes the f-type iterations. Now the proof is by contraction. If the result is false, there should have been a constant and a subsequence of such that
for all . Without loss of generality, we can suppose there are only f-type iterations for . Moreover, there exist the following results for sufficiently large .
As , it is apparent that hold for large enough . It is reasonable that we have from (3.22), so similar to the Lemma 3.9, we can get for large enough . Without loss of generality, suppose for all .
Because and do not change in the f-type iteration, then for all we have and . Since the f-type point satisfies the reduction condition (2.7), that is, , then it holds that By is decreasing for and Lemma 3.8, we get which is a contraction.
If the gradients of are linear independent for all and , then the solution to (1.1) is obtained by virtue of the result in . Thus, the result is established.
Lemma 3.11. If for a positive constant independent of , then there exists a constant , for all and such that
Lemma 3.12. Under all existing assumptions, if for a positive constant independent of and for all with , then there exist , such that for all and .
Proof. Choose , then implies that . So from (3.3), we obtain
And choose , then implies that . It follows from (3.2) that
We further point out a fact according to the definition of filter. If and , , we note that . So from , and , we get .
Lemma 3.13. If there exists such that for it holds for a constant , then there exists so that for all the f-type iterate is always satisfied.
Proof. By Lemma 3.9, there exists and a constant , for such that . From the definition of , when , we have . As , we have . So there exists (>) such that for it holds
which implies for we have .
Because , there exists (>) such that for it follows which shows that Moreover, implies for all .
Let , by Lemmas 3.11 and 3.12, for we obtain If we now denote with the first trial size satisfying (3.32), the backtracking line search procedure then implies that for from which it follows that This means that and all previous trial step sizes are f-step sizes and larger than . Therefore, is the accepted step size indeed. Since the trial point satisfies the switching condition (2.6) and the reduction condition (2.7), the iteration is a f-type iteration for each . The claim is true.
Theorem 3.14. Suppose that is an infinite sequence generated by Algorithm 2.1 and , then Namely, it has an accumulation which is the solution to (1.1) or a local infeasible point. If the gradients of are linear independent for all and , then the solution to (1.1) is obtained.
Proof. Suppose by contraction that there should have been a constant such that
for all . Furthermore, there exists the following results for sufficiently large .
As , it is apparent that holds for large enough . It is reasonable that we have for large enough from (3.36). Therefore there exists such that for it holds for a constant , then by Lemma 3.13, we know that there exists such that for all the f-type iterate is always satisfied, which is a contraction to .
Similar to Theorem 3.10, if the gradients of are linear independent for all and , then the solution to (1.1) is also obtained. The whole proof is completed.
4. Numerical Examples
In this section, we develop the implementation of Algorithm 2.1 in order to observe its performance on some illustrative examples. In the whole process, the program is coded in MatLab with exact line search. The first example is from , which converges to a nonstationary point if the least squares approach is employed. The second example comes from , which Newton method fails to solve, and the final problem is given from . In the tables, the notations NIT, NOF, and NOG mean the number of iterates, number of functions and number of gradients, respectively.
Example 4.1. Find a solution of the nonlinear equations system:
The unique solution is . It has been proved in  that, under initial point , the iterates converge to the point , which is not a stationary point. Utilizing our algorithm, a sequence of points converging to is obtained. We assume the error tolerance in this paper is always . The detailed numerical results for Example 4.1 are listed in Table 1.
Example 4.2. Find a solution of the nonlinear equations system as follows:
The only solution of Example 4.2 is . Define the line . If the starting point , the Newton method  is confined to . We choose two starting points which are belong to in the experiments and then the is obtained. The numerical results of Example 4.2 are given in Table 2.
Example 4.4. Consider the system of nonlinear equations:
with the with the initial point , . The solution to Example 4.4 is . The numerical results of Example 4.4 are given in Table 4.
All results summarized show that our proposed algorithm is practical and effective.
The authors would like to thank the anonymous referees for the careful reading and helpful comments, which led to an improved version of the original paper. The authors gratefully acknowledge the National Natural Science Foundation of China (no. 11126060) and the Science and Technology Program of Shanghai Maritime University (no. 20120060).
- J. Nocedal and S. J. Wright, Numerical Optimization, Springer, New York, NY, USA, 1999.
- M. J. D. Powell, “A hybrid method for nonlinear equations,” in Numerical Methods for Nonlinear Algebraic Equations, P. Rabinowitz, Ed., pp. 87–114, Gordon and Breach, London, UK, 1970.
- R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Mathematical Programming, vol. 91, no. 2, pp. 239–269, 2002.
- R. Fletcher, S. Leyffer, and P. L. Toint, “On the global convergence of a filter-SQP algorithm,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 44–59, 2002.
- R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A. Wächter, “Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 635–659, 2002.
- S. Ulbrich, “On the superlinear local convergence of a filter-SQP method,” Mathematical Programming, vol. 100, no. 1, pp. 217–245, 2004.
- C. Audet and J. E. Dennis, Jr., “A pattern search filter method for nonlinear programming without derivatives,” SIAM Journal on Optimization, vol. 14, no. 4, pp. 980–1010, 2004.
- C. M. Chin and R. Fletcher, “On the global convergence of an SLP-filter algorithm that takes EQP steps,” Mathematical Programming, vol. 96, no. 1, pp. 161–177, 2003.
- M. Ulbrich, S. Ulbrich, and L. N. Vicente, “A globally convergent primal-dual interior filter method for nonconvex nonlinear programming,” Mathematical Programming, vol. 100, pp. 379–410, 2003.
- R. Fletcher and S. Leyffer, “A bundle filter method for nonsmooth nonlinear optimization,” Tech. Rep. NA/195, Department of Mathematics, University of Dundee, Scotland, UK, 1999.
- E. Karas, A. Ribeiro, C. Sagastizábal, and M. Solodov, “A bundle-filter method for nonsmooth convex constrained optimization,” Mathematical Programming, vol. 116, no. 1-2, pp. 297–320, 2009.
- Y. Zhou, “A smoothing conic trust region filter method for the nonlinear complementarity problem,” Journal of Computational and Applied Mathematics, vol. 229, no. 1, pp. 248–263, 2009.
- K. Su and D. Pu, “A nonmonotone filter trust region method for nonlinear constrained optimization,” Journal of Computational and Applied Mathematics, vol. 223, no. 1, pp. 230–239, 2009.
- N. I. M. Gould, S. Leyffer, and P. L. Toint, “A multidimensional filter algorithm for nonlinear equations and nonlinear least-squares,” SIAM Journal on Optimization, vol. 15, no. 1, pp. 17–38, 2004.
- N. I. M. Gould, C. Sainvitu, and P. L. Toint, “A filter-trust-region method for unconstrained optimization,” SIAM Journal on Optimization, vol. 16, no. 2, pp. 341–357, 2005.
- A. Wächter and L. T. Biegler, “Line search filter methods for nonlinear programming: motivation and global convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 1–31, 2005.
- A. Wächter and L. T. Biegler, “Line search filter methods for nonlinear programming: local convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 32–48, 2005.
- P.-Y. Nie, “A null space method for solving system of equations,” Applied Mathematics and Computation, vol. 149, no. 1, pp. 215–226, 2004.
- P.-Y. Nie, “CDT like approaches for the system of nonlinear equations,” Applied Mathematics and Computation, vol. 172, no. 2, pp. 892–902, 2006.
- P.-Y. Nie, “An SQP approach with line search for a system of nonlinear equations,” Mathematical and Computer Modelling, vol. 43, no. 3-4, pp. 368–373, 2006.
- P.-Y. Nie, M.-y. Lai, S.-j. Zhu, and P.-a. Zhang, “A line search filter approach for the system of nonlinear equations,” Computers & Mathematics with Applications, vol. 55, no. 9, pp. 2134–2141, 2008.
- C. Gu, “Nonmonotone filter DQMM method for the system of nonlinear equations,” Applied Mathematics and Computation, vol. 217, no. 22, pp. 9351–9357, 2011.
- P.-Y. Nie, “Sequential penalty quadratic programming filter methods for nonlinear programming,” Nonlinear Analysis, vol. 8, no. 1, pp. 118–129, 2007.
- R. H. Byrd, M. Marazzi, and J. Nocedal, “On the convergence of Newton iterations to non-stationary points,” Mathematical Programming, vol. 99, no. 1, pp. 127–148, 2004.
- H. Schichl and A. Neumaier, “Exclusion regions for systems of equations,” SIAM Journal on Numerical Analysis, vol. 42, no. 1, pp. 383–408, 2004.
Copyright © 2012 Zhong Jin and Yuqing Wang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.