Abstract
We present a filter trust region method for nonlinear semi-infinite programming. Based on the discretization technique and motivated by the multiobjective programming, we transform the semi-infinite problem into a finite one. Together with the filter technique, we propose a modified method that avoids the merit function. Compared with the existing methods, our method is more flexible and easier to implement. Under some mild conditions, the convergent properties are proved. Moreover, the numerical results are reported in the end.
1. Introduction
Consider nonlinear semi-infinite programming problem (SIP) as follows:where is a compact set, is a closed set, is a twice continuously differentiable function, and is continuous and differentiable about variables and . For convenience, consider only the case of ; and are real numbers.
The SIP problem arises in various applications such as approximation theory, optimal control, eigenvalue computation, mechanical stress of materials, pollution control, and statistical design. The research of SIP can be traced back to linear semi-infinite programming [1] and it was also proposed in researching Fritz-John conditions [2]. Then the SIP was formally proposed by Charnes, Coopper, and Kortanek in 1962 [3]. It is difficult to solve SIP because the constraints in SIP are infinite. Numerical methods for solving SIP may be divided into discretization methods and continuous methods. Discretization methods are based on nonlinear programming problems which are obtained by discretization of the original problem and incorporate some grid-refinement strategy [4, 5]. Our new method might be useful in the discretization context for semi-infinite programming since it drastically decreases the number of constraints.
For continuous methods, Hettich and Honstede present an iterative method which attempts to find a solution satisfying optimality conditions of the original problem. However, this method is only locally convergent and is very restrictive for practical use [6]. Jian et al. present a Norm-Relaxed Method of Feasible Directions algorithm, in order to obtain global convergence [7]. Coope and Watson have proposed a Sequential Quadratic Programming (SQP) method that utilizes an exact penalty function and global convergence obtained [8]. Then, a globally convergent SQP method for SIP is proposed which utilizes an exact penalty function and trust region methods [9]. For optimization problems with smooth inequality constraints, the penalty function method is, in general, recognized as an efficient method. In [10, 11], in order to solve SIP, the smooth approximate functions in integral form are appended to the objective function by using the concept of the penalty function, but the error caused by taking the smooth approximation of the continuous inequality constraints cannot be avoided. Moreover, it is difficult to find a suitable penalty parameter. If the penalty parameter is too large, then any monotonic method would be forced to follow the nonlinear constraint manifold very closely, resulting in much shortened Newton steps and slow convergence.
To avoid using the above drawbacks, Fletcher and Leyyfer proposed a filter method in 2002 [12]. In their method, instead of combining the objective and constraint violation into a single function, they view the original problem as a biobjective optimization problem that minimizes objective function and constraint violation function. Consequently, filter technique has been employed in many approaches, for instance, SQP methods [13], interior point approaches [14], bundle techniques [15], and SIP [16]. But, in [16], the search direction is obtained in a linear equation. The structure of filter is complex.
Motivated by the above ideas, we propose a filter trust region algorithm. We transform the SIP into a finite problem based on a modified discretion method. Then, the search direction is obtained by the modified trust region quadratic subproblem. Moreover, without the merit function, we adopt the filter technique to decide whether a new iteration point is acceptable or not. Compared with the existing methods, there are two main advantages in our presented algorithm: (1) the modified quadratic subproblem is always feasible; (2) unlike the continuous method, the penalty parameter is avoided in our presented method, and actually the algorithm has a certain nonmonotonicity property. Under some wild conditions, the convergent properties of the proposed method are proved. The numerical results show the effectiveness of our approach.
The remainder of this paper is organized as follows. The algorithm is described in Section 2. In Section 3, we analyze the convergent properties of proposed algorithm. Numerical results are reported in the final section.
2. Description of Algorithm
For solving SIP, discretization of the original problem SIP, consider the following problem:where is the discrete subset of that appeared in SIP problem (1). is a positive integer (in general, ) that represents the discrete level. In practice, depending on the length of [a, b], and are real numbers, suppose .
For any , denote ,
, ,
, , , .
Suppose the current iteration point is , , , . .
In order to improve the degree of accuracy of optimal solution of SIP, we observed that trust region method is exceedingly important for ensuring global convergence while retaining fast local convergence in optimization algorithms, so the trust region condition is added to the quadratic subproblem of (2).
For the current k-th iteration, is obtained by the following modified trust region quadratic subproblem (QP):where are positive constants, is a symmetric positive definite matrix, and is the k-th trust region radius.
DenoteDefine the actual reduction of asThe predict reduction of asHere .
DefineThe parameter is used to decide whether the predicted reduction of is close to the actual reduction of or not.
In the proposed algorithm, the filter technique is used to decide whether a trial point is acceptable or not.
Definition 1. A pair is dominated by if and only if and for each .
Definition 2. A filter set is a set of pairs such that no pair dominates any other.
To ensure the convergence, some additional conditions are required to decide whether to accept a trial point to the filter or not. The traditional acceptable criterion is as follows.
Definition 3. A trial point is called acceptable to the filter if and only if either
To avoid convergence to infeasible limit points, we add an envelope around the current filter. So criterion (8) can be changed to the following criterion.
Definition 4. A trial point is called acceptable to the filter if and only if eitherwhere are constants. In practice, is close to and is close to .
In the current k-th iteration, let . Then our algorithm (FTR) is presented as follows.
FTR Algorithm
Step 1 (initialization). Given , , and set k=0, , , , , , , , and .
Step 2. Solve (3) to obtain If , stop; otherwise, set , and go to Step 2.
Step 3. Calculate , where .
Step 4 3. If , set k:=k+1, and update parameter and go to Step 1; otherwise go to Step 4.
Step 5 4. If is acceptable for current filter, add the to filter, set k:=k+1, , update parameter , and go to Step 1; otherwise, set k:=k, and go to Step 1.
Suppose that QP has local optimal solution ; i.e., is a Karush-Kuhn-Tucker (KKT) point of QP. So there exists multiplier and , satisfy:
Similarly, if is a KKT point for problem , then there exists a multiplier vector and such that the following formulas hold:
The following lemma shows that our algorithm is well defined.
Assumption A1. and are continuous and differentiable.
Lemma 5. Assume Assumption A1 holds and ; then (i) subproblem (QP) has a unique solution; (ii) is an optimal solution of subproblem (QP) if and only if is a Karush-Kuhn-Tucker (KKT) point of it.
Proof. (i) Obviously, is a feasible solution of subproblem (QP) since for all as well as for all ; beside also satisfy and , so the feasible set of subproblem (QP) is not empty. On the other hand, for each feasible solution of subproblem (QP), in view of the first constraint of subproblem (QP), we know that the value of objective function of subproblem (QP) satisfies the following inequality:Therefore, the value of objective function of subproblem (QP) is bounded from below because of the symmetric positive definite property of ; that is, there exists a constant such thatHere the feasible setAccording to Bolzano-Weierstrass theorem, bounded series must have a convergent subsequence. Therefore, there exists a sequence such thatMoreover, for large enough, we haveNote that the matrix is positive definite, as well as the boundedness of , so the sequence is bounded. Therefore, there exists a subset such thatHence is an optimal solution of subproblem (QP). In addition, the subproblem (QP) is equal to the following constrained optimization:Obviously, the first term of the objective function is strictly convex about variable and the second term is convex about variable ; thus the objective function of the problem above is strictly convex [18]. Besides, the constrained function of (31) is a convex function. Combining the convexity of , we obtain problem (31) which is a convex programming and its optimal solution is unique. Therefore, the optimal solution of subproblem (QP) is unique.
(ii) If is a KKT point for subproblem (QP), then it is an optimal solution of subproblem (QP) since subproblem (QP) is a convex programming. Conversely, if is an optimal solution of subproblem (QP), note that Abadie’s Constraint Qualification [19] holds automatically since the constraints of subproblem (QP) are linear; then is a KKT point for subproblem (QP). The proof is completed.
From Lemma 5, we know QP has only optimal solution ; i.e., QP has only KKT point. So there exists multiplier and , satisfy (10)-(20).
Similarly, if is a KKT point for problem , then there exists a multiplier vector and satisfy (21)-(24).
Assumption A2. Assume the Mangasarian-Fromovitz Constraint Qualification (MFCQ) holds at any ; i.e., there exists a vector such that .
Lemma 6. Assume Assumptions A1 and A2 hold, is an optimal solution of subproblem (QP); then (i) ; (ii) if and only if if and only if is a KKT point for subproblem (QP).
Proof. (i) From the fact that is a feasible solution of subproblem (QP) and is an optimal solution of subproblem, then Besides is positive definite, and ; one has .
(ii) From (i), we have . If , then ; taking into account the positive definite property of , one has . Conversely, if , we denote the feasible set of problem :and two cases and are considered, respectively. Firstly, suppose that , i.e., , from the first constraint of subproblem , we have . Combining with the result that and , we know that the conclusion holds. Now suppose that , i.e., , then the set and let such that . So from the constraints of subproblem (QP) one gets Together with , one gets . Conclusion (ii) is obtained.
Lemma 7. Assume and , and is defined as (32); then the set .
Proof. In view of the definition of and , it follows that . Now we prove by contradiction that the first conclusion of Lemma 7 holds. Suppose that there exists a vector such thatthe following result holds if the positive parameter is small enoughObviously, for small enough, is a feasible solution of (3); furthermore, ThereforeThis is a contradiction to . Hence Lemma 7 follows. The proof is completed.
Lemma 8. Assume Assumptions A1 and A2 hold; if , is defined as (32), then .
Proof. By contradiction, suppose that does not hold; then , and we have from Lemma 7; this conclusion contradicts to Assumption A2. The proof is completed.
Lemma 9. Assume Assumptions A1 and A2 hold. Then the multiplier sequence and are both bounded.
Proof. In view of the formula and the nonnegative properties of multipliers and parameter , we can obtain the boundedness of sequences and .
Lemma 10. Assume Assumptions A1 and A2 hold and is an optimal solution of subproblem (QP); then if and only if is a KKT point of .
Proof. Suppose that , we can get , by Lemma 6, so , , and the KKT conditions (10)-(20) can be rewritten asIn view of the formula and the nonnegative properties of multipliers and parameter , we can obtain the boundedness of sequences and . We prove that . By contradiction, suppose that , then in view of the equation one can get ; i.e., , one knows that from . Therefore, becomesChoosing a vector satisfying MFCQ at the point and multiplying the equality above and the constraints of subproblem for by , we obtainThis is a contradiction and the conclusion that is true.
Let and then the KKT condition of holds at with the multiplier vector , so is a KKT point for .
Now, to prove the necessity of theorem, note that if is a KKT point for , namely, (37)-(41) holds, then satisfies KKT conditions (37)-(41) of with multipliersFrom Lemma 5 we obtain the uniqueness of the KKT point. So the uniqueness of the KKT point shows that . The whole proof is completed.
From Lemma 10, can be used as terminating condition.
Lemma 11. FTR algorithm cannot cycle infinitely many times among the inner cycle (Step 1 Step 2 Step 4 Step 1).
Proof. Suppose the contrary, then it follows from the rules of FTR algorithm that one has , while is maintained. Consequently, we havesoThis indicates that we eventually have . This is a contradiction and the desired conclusion follows. This proof is completed.
3. Convergent Properties
In this section, under appropriate conditions, we establish the convergence of FTR algorithm. For this purpose, the following basic assumption is necessary.
Assumptions A3. Assume the sequence generated by FTR algorithm is bounded.
Assumptions A4. Assume there exists , and , such that .
To analyze the convergence property of FTR algorithm, we need the following lemma.
Lemma 12. Assume Assumptions A3 and A4 hold; then, sequences and generated by FTR algorithm are bounded.
Proof. Firstly, prove is bounded. By the contradiction, suppose is unbounded, there exists infinite index set , such that as . Together with , this is a contradiction with , so is bounded.
Now, because of is bounded and is continuous and differentiable, thus is bounded. According to and one has furthermore, together with , is bounded. This proof is completed.
The following result is obtained.
Theorem 13. Assume Assumptions A3 and A4 hold, is an infinite iterative index set; if , then any accumulation point of , generated by FTR algorithm is the KKT point of .
Proof. Suppose is the accumulation point of . Denote is the subset of .
First, we prove as .
If , then ; besides, , , so ; thus as .
If , then ; there exists such that . Thus as .
Then, from Lemmas 9 and 10, the conclusion is obtained.
Because we use the filter technique in FTR algorithm, the following lemma is necessary.
Lemma 14. Assume there are infinitely many points added to the filter. Then
Proof. If the theorem is not true, there would have infinite components in , which is defined as follows:We assume that for all without loss of generality, where is a positive constant. The following two cases are considered, respectively.
If exists, let , , and . Then, according to the definition of the filter, the other components, which lie behind in the filter, satisfyThen, all the filter points, which enter the filter behind can be covered with a square, whose area is no more than . We consider the area lies to the southwest of the filter in this square. When a new point enters the filter, the next point should lie to the southwest of the points in the filter and the area which lies to the southwest of in the square is smaller than that of . Therefore, we think that the area is reduced if a new point enters the filter. When a point is added to the filter, its is less than every point, which lies to the left of this point, to more than . Its is less than every point, which lies to the right of this point, to more than . Therefore, the area of this square, more than , will be reduced. Thus, the area will be reduced to 0 after finite times. When the area is zero, it means that a point does not enter , which is contradicted with the infiniteness of .
(ii) If does not exist, because and are continuous and differentiable, letFrom the definition of , there exists such that and Then, according to the definition of the filter, the other components, which lie behind in the filter, satisfyUsing the same techniques as that in (i), the conclusion can also obtained which is contradicted with the infiniteness of . So is infinity; the result is gotten.
Thus, the conclusion is obtained.
As for the case of finitely many points added to the filter, it is apparent that the following result is true.
Lemma 15. Assume there are finitely many points added to the filter. Then , where is the number of points added to the filter.
Proof. From the terminating condition of Step 2 in FTR algorithm, one has and also ; furthermore . Thus, if there are finitely many points added to the filter, the index of the last point is the number of points added to the filter. The proof is completed.
The following result is obtained.
Theorem 16. Assume Assumptions A3 and A4 hold; if generated by FTR algorithm is a bounded sequence, then has an accumulation point satisfying the KKT conditions of .
Proof. Suppose is an accumulation point of sequence , by Lemma 14, , so there exists such that . Taking the limits about KKT conditions (10)-(20) of subproblem and , together with Lemma 9, we have , so ; thus is the KKT point of . The proof is completed.
4. Numerical Results
In this section, we report our preliminary numerical test results. FTR algorithm described in Section 2 was then implemented in Matlab R2016a. We compared the performance of the FTR algorithm with [17]. The following four problems were tested. Throughout the computational experiments, the parameters used in algorithm were , , , , , , and .
Problem 1.
Problem 2.
Problem 3.
Problem 4.
In Table 1, the problem is denoted by Pro; e.g., Problem 1 is denoted by Pro1, the dimension of is denoted by n, the number of iterations is denoted by Iter, the optimal solution is denoted by , and the optimal value is denoted by .
In Table 1, the limited numerical results show that the proposed algorithm can give an accurate solution quickly (the iterate number is less than that in [17]), which indicates that the algorithm herein is effective. Moreover, it has been observed that the computational results are not very sensitive to the choice of the initial point and the parameter n, showing that the proposed algorithm is stable.
5. Conclusion
In this paper, we present a filter trust region method. Based on the discrete technique, we transform nonlinear semi-infinite programming into a finite problem. And the search direction is obtained by a modified trust region quadratic subproblem which is always feasible. Besides, we adopt the filter technique to decide whether a new iteration point is acceptable or not, so the penalty parameter which is always used in most of the existing methods is avoided. Under some wild conditions, the global convergence of the proposed method was obtained. The main advantage of the presented method is that our method is more flexible and easier to implement, the modified trust region quadratic subproblem is always feasible, and penalty parameter is avoided. The numerical results show that our presented method is effective and stable.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (no. 61572011), the Natural Science Foundation of Hebei Province (no. A2018201172), and the Key Research Foundation of Education Bureau of Hebei Province (no. ZD2015069).