- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Mathematical Problems in Engineering
Volume 2012 (2012), Article ID 349178, 20 pages
A Filter Algorithm with Inexact Line Search
1Department of Mathematics, Tongji University, Shanghai 200092, China
2Business School, University of Shanghai for Science and Technology, Shanghai 200093, China
3School of Management, Fudan University, Shanghai 200433, China
Received 25 October 2011; Accepted 18 December 2011
Academic Editor: Dongdong Ge
Copyright © 2012 Meiling Liu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
A filter algorithm with inexact line search is proposed for solving nonlinear programming problems. The filter is constructed by employing the norm of the gradient of the Lagrangian function to the infeasibility measure. Transition to superlinear local convergence is showed for the proposed filter algorithm without second-order correction. Under mild conditions, the global convergence can also be derived. Numerical experiments show the efficiency of the algorithm.
Fletcher and Leyffer  proposed filter methods in 2002 offering an alternative to traditional merit functions in solving nonlinear programming problems (NLPs). The underlying concept is that a trial point is accepted provided that there is a sufficient decrease of the objective function or the constraint violation. Filter methods avoid the difficulty of determining a suitable value of the penalty parameter in the merit function. The promising numerical results in  led to a growing interest in filter methods in recent years. Two variants of trust-region filter sequential quadratic programming (SQP) method were proposed by Fletcher et al. [2, 3]. Chin and Fletcher  developed filter method to sequential linear programming strategy that takes equality-constrained quadratic programming steps. Ribeiro et al.  proposed a general filter algorithm that does not depend on the particular method used for the step of computation. Ulbrich  argued superlinear local convergence of a filter SQP method. Ulbrich et al.  and Wächter and Biegler  applied filter technique to interior method and achieved the global convergence to first-order critical point. Wächter and Biegler [9, 10] proposed a line-search filter method and applied it to different algorithm framework. Gould et al.  and Shen et al.  developed new multidimensional filter technique. Su and Pu  extended the monotonicity of the filter technique. Nie  applied filter method to solve nonlinear complementarity problems. In this paper, the global convergence is analyzed widely. However, it has been noted by Fletcher and Leyffer  that the filter approach can suffer from the Maratos effect as that of a penalty function approach. By the Maratos effect, a full step can lead to an increase of both infeasibility measure and objective function in filter components even if arbitrarily close to a regular minimizer. This makes the full step unacceptable for the filter and can prohibit fast local convergence.
In this paper, we propose a filter algorithm with inexact line-search for nonlinear programming problems that ensures superlinear local convergence without second-order correction steps. We use the norm of the gradient of the Lagrangian function in the infeasibility measure in the filter components. Moreover, the new filter algorithm has the same global convergence properties as that of the previous works [2, 3, 9]. In addition, since the sufficient decrease conditions in an SQP framework can usually make the algorithm complex and time-consuming, the presented method is a line-search method without using SQP steps. An inexact line-search criterion is used as the sufficient reduction conditions. In the end, numerical experiences also show the efficiency of the new filter algorithm.
This paper is organized as follows. For the main part of the paper, the presented techniques will be applied to general NLP. In Section 2, we state the algorithm mechanism. The convergent properties are shown in Section 3. The global and superlinear convergence are proved. Furthermore, the Maratos effect is avoided. Finally, Section 4 shows the effectiveness of our method under some numerical experiences.
2. Inexact Line-Search Filter Approach
2.1. The Algorithm Mechanism
We describe and analyze the line-search filter method for NLP with equality constraints. State it as where the objective function and the constraints are assumed to be continuously differentiable, and .
The corresponding Lagrangian function is where the vector corresponds to the Lagrange multiplier. The Karush-Kuhn-Tucker (KKT) conditions for (2.1) are
For a given initial estimate , the line-search algorithm generates a sequence of iterates by as the estimates of the solution for (2.1). Here, the search direction is computed from the linearization at of the KKT conditions (2.3): where the symmetric matrix denotes the Hessian of (2.2) or a positive definite approximation to it.
After a search direction has been computed, the step size is determined by a backtracking line-search procedure, where a decreasing sequence of step size is tried until some acceptable criteria are satisfied. Generally, the acceptable criteria are constructed by a condition that if the current trial point can provide sufficient reduction of a merit function. The filter method proposed by Fletcher and Leyffer  offers an alternative to merit functions. In this paper, the filter notion is defined as follows.
Definition 2.1. A pair is said to dominate another pair if and only if both and .
Here, we define as the norm of the infeasibility measure. That is, we modify the infeasibility measure in filter, with this modification, the superlinear convergence is possibly derived. Strictly, it is a stationarity measure. However, we still call it infeasibility measure according to its function. In the rest of paper, the norm is always computed by norm excepting special noting.
Definition 2.2. A filter is a list of pairs such that no pair dominates any other. A point is said to be acceptable for inclusion in the filter if it is not dominated by any point in the filter.
When a pair is said to be acceptable to the filter, we also say the iterate is acceptable to the filter. In filter notion, a trial point is accepted if it improves feasibility or improves the objective function. So, it is noted that filter criteria is less demanding than traditional penalty function. When improving optimality, the norm of the gradient of the Lagrangian function will tend to zero, so it offers a more precise analysis for the objective function.
However, this simple filter concept is not sufficient to guarantee global convergence. Fletcher et al.  replace this condition by requiring that the next iterate provides at least as much progress in one of the measure or that corresponds to a small fraction of the current infeasibility measure. Here, we use the similar technique to our filter. Formally, we say that a trial point can be accepted to the current iterate or the filter if orfor some fixed constants , and are points in current filter. In practical implementation, the constants close to 1 and close to 0. However, the criteria (2.5a) and (2.5b) may make a trial point always provides sufficient reduction of the infeasibility measure alone, and not the objective function. To prevent this, we apply a technique proposed in Wächter and Biegler  to a different sufficient reduction criteria. The switching condition is where , , . If the condition (2.6) holds, we replace the filter condition (2.5b) as an inexact line-search condition, that is, the Armijo type condition where is a constant. If (2.6) holds but not (2.7), the trial points are still determined by (2.5a) and (2.5b).
If a trial point can be accepted at a step size by (2.7), we refer to as an type iterate and the corresponding as an step size.
2.2. The Algorithm Analysis
By the right part of switching condition (2.6), it ensures that the improvement to the objective function by the Armijo condition (2.7) is sufficiently large compared to the current infeasibility measure . Thus, if iterate points remote from the feasible region, the decrease of the objective function can be sufficient. By setting , the progress predicted by the line model of can be a power of the infeasibility measure . The choice of makes it possible that a full step can be accepted by (2.7) when it closes to a local solution.
In this paper, we denote the filter as a set containing all iterates accepted by (2.5a) and (2.5b). During the optimization, if the type switching condition (2.6) holds and the Armijo condition (2.7) is satisfied, the trial point is determined by (2.7) not by (2.5a) and (2.5b), and the value of the objective function is strictly decreased. This can prevents cycling of the algorithm (see ).
If the linear system (2.4) is incompatible, no search direction can be found and the algorithm switches to a feasibility restoration phase. In it, we try to decrease the infeasibility measure to find a new iterate that satisfies (2.4) and is acceptable to the current filter. Similarly, in case , the sufficient decrease criteria for the objective function in (2.5b) may not be satisfied. To inspect where no admissible step size can be found and the feasibility restoration phase has to be invoked, we can consider reducing and define If the trial step size , the algorithm turns to the feasibility restoration phase.
By , it is ensured that the algorithm does not switch to the feasibility restoration phase as long as (2.6) holds for a step size and that the backtracking line-search procedure is finite. Thus, for a trial point , the algorithm eventually either delivers a new iterate or reverts to the feasibility restoration phase. Once finding a feasible direction, the algorithm still implements the normal algorithm.
Of course, the feasibility restoration phase may not always be possible to find a point satisfying the filter-accepted criteria and the compatible condition. It may converge to a nonzero local minimizer of the infeasibility measure and indicate that the algorithm fails. In this paper, we do not specify the particular procedure for the feasibility restoration phase. Any method for dealing with a nonlinear algebraic system can be used to implement a feasibility restoration phase.
2.3. The Algorithm
We are now in the place to state the overall algorithm.
Algorithm 2.3. Step 1. Given: starting point , constants , , , , , , .Step 2. Initialize: , the iteration counter .Step 3. For , stop if satisfies the KKT conditions (2.3).Step 4. Compute the search direction from (2.4). If the system (2.4) is incompatible, go to the feasibility restoration phase in Step 7.Step 5. Set , compute .(1)If , go to Step 7. Otherwise, compute the new trial point .(2)If the conditions (2.6) and (2.7) hold, accept the trial step and go to Step 6, otherwise set , go to Step 5(3).(3)In case where no make (2.7) hold, if can be accepted to the filter, augment the filter by , go to Step 6; Otherwise set , go to Step 5(4).(4)Compute , go back to Step 5(1).Step 6. Increase the iteration counter and go back to Step 4.Step 7. Feasibility restoration phase: by decreasing the infeasibility of to find a new iterate such that (2.4) is compatible. And if (2.7) holds at , continue with the normal algorithm in Step 6; if (2.5a) and (2.5b) hold at , augment the filter by , and then continue with the normal algorithm in Step 6; if the feasibility restoration phase cannot find such a point, stop with insuccess.
Remark 2.4. In contrast to SQP method with trust-region technique, the actual step does not necessarily satisfy the linearization of the constraints.
Remark 2.5. Practical experience shows that the filter allows a large degree of nonmonotonicity and this can be advantageous to some problems.
Remark 2.6. To prevent the situation in which a sequence of points for which are type iterative point with is accepted, we set an upper bound on the infeasibility measure function .
3. Convergence Analysis
3.1. Global Convergence
In this section, we give a global convergence analysis of Algorithm 2.3. We refer to the global convergence analysis of Wächter and Biegler  in some places. First, state the necessary assumptions.
Assumption A1. Let all iterates are in a nonempty closed and bounded set of .
Assumption A2. The functions and are twice continuously differentiable on an open set containing .
Assumption A3. The matrix is positive definite on the null space of the Jacobian and uniformly bounded for all , and the Lagrange multiply is bounded for all .
For stating conveniently, we define a set . In addition, sometimes, it is need to revise to keep it positive definite by some updating methods such as damped BFGS formula  or revised Broyden’s method .
Lemma 3.1 shows that the search direction is a descent direction for the objective function when the trial points are sufficiently close to feasible region.
Proof. From Assumptions A1 and A2, we know that is bounded. Hence, it has with (3.1) that there exists a constant such that By (2.6) it has As , we have Then by (2.7) and (3.7), Hence, for , an integer and all , Since is bounded below as , the series on the right hand side in the last line of (3.8) is bounded, then implies the conclusion.
Lemma 3.3. Let be an infinite subsequence of iterates so that is entered into the filter. Then
Proof. Here, we refer to the proof of [2, Lemma 3.3]. If the conclusion is not true, there exists an infinite subsequence such that for all and for some . This means that no other pair can be added to the filter at a later stage within the region or with the intersection of this region with for some constants . Now, the area of each of these regions is . Hence, the set is completely covered by at most a finite number of such regions, for any . Since the pairs keep on being added to the filter, tends to infinity when tends to infinity. Without loss of generality, assume that for all is sufficiently large. But (2.5a) and (3.11) imply that so , which contradicts (3.11). Then, this latter assumption is not true and the conclusion follows.
Next, we show that if is bounded, there exists at least one limit point of the iterative points is a first-order optimal point for (2.1).
Proof. Since the filter is augmented only a finite number of times, there exists an integer so that for all iterates the filter is not augmented. If the claim is not true, there must exist a subsequence and a constant so that for all . Then by Lemma 3.1, it has for all , is some integer and . And from Lemmas 3.2 and 3.4, it has and Since is bounded below and monotonically decreasing for all , one can conclude that . This means that for the step size has not been accepted. So, we can get a such that a trial point satisfies or Let . From Lagrange’s Theorem, it has for some constant , where denotes some point on the line segment from to . Since and by Lemmas 3.2 and 3.3, it has for sufficiently large, so (3.19) is not true. In case (3.20), since for sufficiently large , we have with from Lemma 3.4, that is, (3.20) can not be satisfied. Then the conclusion follows.
Proof. The mechanisms of Algorithm 2.3 ensure that the first iterate can be accepted to the filter. Next, we can assume that is acceptable to the th filter and . If , it has that is, , so by (3.16) Similarly, if , with and from Lemma 3.5, it has Hence, we have The claim then follows from (3.25).
The last Lemma 3.6 shows, for case , Algorithm 2.3 either accepts a new iterate to the filter or switches to the feasibility restoration phase. For case and the algorithm does not stop at a KKT point, then , , and the Armijo condition (2.7) is satisfied for sufficiently small step size , so an type iterate is accepted. Hence, the inner loop in Step 5 always terminates in a finite number of trial steps, and Algorithm 2.3 is well defined.
Lemma 3.7. Suppose Assumptions A1–A3 hold. Let be a subsequence with for a constant independent of . Then, there exists an sufficient large integer such that for all the algorithm can generate some trial points either be accepted to the filter or be type steps.
Proof. By Lemmas 3.1, 3.2, and 3.3, there exist constants so that
for all .
If , the type switching condition (2.6) is true, there must exist iterates for which are type iterates. For the remaining iterates with , if with from Lemma 3.4 and from Lemma 3.6, it implies with as well as Now choose an arbitrary with and define Lemmas 3.4 and 3.6 then imply that a trial step size satisfies both
Since by the definition of , the method does not switch to the feasibility restoration phase for those trial step sizes. Then the claim follows.
Based on the above lemmas, we can give the main global convergence result.
Proof. (3.32) follows from Lemmas 3.2 and 3.3.
If the filter is augmented a finite number of times, then the claim (3.33) holds from Lemma 3.5. For either case, there exists a subsequence so that for all . Suppose the conclusion (3.33) is not true, there must exist a subsequence of such that for some constant . Hence by Lemmas 3.1 and 3.3, it has and for all . Then by Lemma 3.7, when , the algorithm can generate a type iterate, that is, the filter is not augmented, this contradicts the choice of , so that (3.33) holds.
3.2. Local Convergence
In this section, we show the local convergence of Algorithm 2.3.
Assumption A4. The iterates converge to a point that satisfies
Assumption A5. There is a neighborhood of such that , for all .
Remark 3.10. Under Assumptions A4 and A5, it is well known that with the choice , the sequence converges -superlinearly to and that the convergence is -quadratic if and are lipschitz continuous in a neighborhood of . That is, for any given , , , and , it has
We use the proof techniques of local convergence in . In proof, define and with is a parameter.
Lemma 3.11. Suppose Assumptions A1–A3 hold. Let satisfy the Assumption A4. Then, there exist constants , and a neighborhood such that
for all and all .
Moreover, possibly after increasing and reducing , we have that for all and all
Proof. Let . Using Taylor’s Theorem and , we have with some on the line segment between and
Obviously, it has
Under Assumption A4, there exists such that for all ,
with a constant , see [15, Theorem 17.5].
Suppose is Lipschitz continuous and is the Lipschitz constant of , and is a constant. Let for all with , it has by continuity Thus, we obtain for all by (3.38), (3.39), and (3.41) It is obvious for all that If , , then it has Since , , and by (3.44) it has that is the left inequality in (3.36). For the right inequality in (3.36), it is obvious from (3.38) that for all , with a constant . This proves the inequality (3.36).
Choose large enough such that for , . We can assume . Then, if is large enough, it has from (3.44) and that By an analogue of (3.45) holds for , this proves the left inequality in (3.37).
On the other hand, it has Since and are twice continuously differentiable on closed set , we have and . This shows the right inequality in (3.37) possibly after increasing .
Proof. Let as in Assumption A5, and be given by Lemma 3.11. For all , is a sufficient large integer, and choose so large that Then, it has from (3.37) in Lemma 3.11 Let By and continuity there exists such that for all , so the point is acceptable to . Since , for all , is an integer. By (3.35), we can choose so small that for all , the sequence with converges linearly with a contraction factor of at least Suppose an arbitrary such that (3.49) holds, and set . By (3.54), it has and is acceptable to . Next, it is a need to show that is acceptable with respect to for . By (3.49), it has , so by (3.52) And, by (3.52) and (3.54) Next, suppose with is not acceptable to then Thus, it has with (3.51) and (3.52) This shows with (3.55) that This produces a contradiction to (3.56), so is acceptable to . Then, the acceptability of for follows by induction.
Next, we show that the sequence with can make the sufficient decreasing condition (2.7) hold.
Proof. Suppose and hold. By , thus
On the other hand, with the assertion yields Thus, where on the line segment between and .
Obviously, we can prove the conclusion if with large enough and for all the following holds
Since , . By Assumption A4, has full-row rank, there exists such that thus,
Choose large enough such that for all . By and (3.66) it has for a constant , we can choose suitable parameters such that the last inequality holds. Thus, This completes the proof.
Theorem 3.14. Suppose Assumptions A1–A5 hold. Then, there exists such that Algorithm 2.3 takes steps with for all , that is, In particular, converges -superlinearly to . If and are Lipschitz continuous in a neighborhood of then converges -quadratically.
Proof. Since Assumptions A4 and A5 hold, with satisfying A4. By Lemmas 3.12 and 3.13, the iterate is acceptable to the filter and satisfies the sufficient decreasing condition (2.7). Thus, the trial iterate is accepted by the algorithm and it has That is in both cases the algorithm takes the steps with . And according to Remark 3.10, converges -superlinearly to .
4. Numerical Experience
In this section, we give some numerical results of Algorithm 2.3. We take some CUTE problems , which are available freely on NEOS, to test our algorithm. The test codes are edited in MATLAB. The details about the implementation are described as follows.(a)The parameters are set to , , , , , , , the termination tolerance .(b)The optimal residual is defined as That is, the algorithm terminates when .(c) is updated by damped BFGS formula .
The detailed results of the numerical test on small-scale problems are summarized in Table 2. For comparison purposes, we also give the numerical results of tridimensional line-search filter solver (Tri-filter) in Shen et al.  in Table 2. The row headers in Tables 2 and 3 are presented in Table 1.
In addition, we also test some mid-scale problems. And we compare the numerical results which are summarized in Table 3 in Algorithm 2.3 and SNOPT solver in Gill et al.  since no mid-scale problems are given in trifilter solver.
From Table 3, we find the efficiency of Algorithm 2.3 is also improved significantly. From both Tables 2 and 3, in general, the behavior of the proposed algorithm is rather stable. Finally, we may conclude that, as far as our limited computational experience is concerned, the proposed algorithm is well comparable to trifilter solver and SNOPT solver.
This research is partially supported by the National Natural Science Foundation of China (Grant nos. 10771162 and 91024021).
- R. Fletcher and S. Leyffer, “Nonlinear programming without a penalty function,” Mathematical Programming, vol. 91, no. 2, pp. 239–269, 2002.
- R. Fletcher, N. I. M. Gould, S. Leyffer, P. L. Toint, and A. Wächter, “Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 635–659, 2002.
- R. Fletcher, S. Leyffer, and P. L. Toint, “On the global convergence of a filter-SQP algorithm,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 44–59, 2002.
- C. M. Chin and R. Fletcher, “On the global convergence of an SLP-filter algorithm that takes EQP steps,” Mathematical Programming, vol. 96, no. 1, Ser. A, pp. 161–177, 2003.
- A. A. Ribeiro, E. W. Karas, and C. C. Gonzaga, “Global convergence of filter methods for nonlinear programming,” SIAM Journal on Optimization, vol. 19, no. 3, pp. 1231–1249, 2008.
- S. Ulbrich, “On the superlinear local convergence of a filter-SQP method,” Mathematical Programming, vol. 100, no. 1, pp. 217–245, 2004.
- M. Ulbrich, S. Ulbrich, and L. N. Vicente, “A globally convergent primal-dual interior-point filter method for nonlinear programming,” Mathematical Programming, vol. 100, no. 2, pp. 379–410, 2003.
- A. Wächter and L. T. Biegler, “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming,” Mathematical Programming, vol. 106, no. 1, pp. 25–57, 2006.
- A. Wächter and L. T. Biegler, “Line search filter methods for nonlinear programming: motivation and global convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 1–31, 2005.
- A. Wächter and L. T. Biegler, “Line search filter methods for nonlinear programming: local convergence,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 32–48, 2005.
- N. I. M. Gould, C. Sainvitu, and P. L. Toint, “A filter-trust-region method for unconstrained optimization,” SIAM Journal on Optimization, vol. 16, no. 2, pp. 341–357, 2006.
- C. G. Shen, W. J. Xue, and D. G. Pu, “Global convergence of a tri-dimensional filter SQP algorithm based on the line search method,” Applied Numerical Mathematics, vol. 59, no. 2, pp. 235–250, 2009.
- K. Su and D. G. Pu, “A nonmonotone filter trust region method for nonlinear constrained optimization,” Journal of Computational and Applied Mathematics, vol. 223, no. 1, pp. 230–239, 2009.
- P. Y. Nie, “A filter method for solving nonlinear complementarity problems,” Applied Mathematics and Computation, vol. 167, no. 1, pp. 677–694, 2005.
- J. Nocedal and S. J. Wright, Numerical Optimization, Springer Series in Operations Research, Springer, New York, NY, USA, 1999.
- D. G. Pu, S. H. Gui, and W. W. Tian, “A class of revised Broyden algorithms without exact line search,” Journal of Computational Mathematics, vol. 22, no. 1, pp. 11–20, 2004.
- I. Bongartz, A. R. Conn, N. Gould, and P. L. Toint, “CUTE: constrained and unconstrained testing environment,” ACM Transactions on Mathematical Software, vol. 21, no. 1, pp. 123–160, 1995.
- P. E. Gill, W. Murray, and M. A. Saunders, “SNOPT: an SQP algorithm for large-scale constrained optimization,” SIAM Review, vol. 47, no. 1, pp. 99–131, 2005.