Abstract
A matrix-free method for constrained equations is proposed, which is a combination of the well-known PRP (Polak-Ribière-Polyak) conjugate gradient method and the famous hyperplane projection method. The new method is not only derivative-free, but also completely matrix-free, and consequently, it can be applied to solve large-scale constrained equations. We obtain global convergence of the new method without any differentiability requirement on the constrained equations. Compared with the existing gradient methods for solving such problem, the new method possesses linear convergence rate under standard conditions, and a relax factor is attached in the update step to accelerate convergence. Preliminary numerical results show that it is promising in practice.
1. Introduction
Let be a continuous nonlinear mapping and a nonempty closed convex set of . In this paper, we consider the problem of finding such that Nonlinear constrained equations (1), denoted by CES (), arise in various applications, for instance, ballistic trajectory computation and vibration systems [1], the power flow equations [2], chemical equilibrium systems [3], and so forth.
In recent years, many numerical methods have been proposed to find a solution of nonsmooth CES (), which include the trust region methods [4, 5], the Levenberg-Marquardt method [6], and the projection methods [7–9]. Compared with the trust region method and the Levenberg-Marquardt method, the projection method is more efficient for solving large-scale CES (). Noting this, Wang et al. [7] proposed a projection method for solving CES (), which possesses global convergence property without the differentiability. A drawback of this method is that it needs to solve a linear equation inexactly at each iteration, and its variants [8, 10] also have this drawback.
It is well-known that the spectral gradient method and the conjugate gradient method are two efficient methods for solving large-scale unconstrained optimization problems due to their simplicity and low storage. Recently, La Cruz and Raydan [11] successfully applied the famous spectral gradient method to solve unconstrained equations by using some merit function. Then, Zhang and Zhou [12] presented a spectral gradient projection method (SGP) for solving unconstrained monotone equations, which does not utilize any merit function. Later, the SGP was extended by Yu et al. [9] to solve monotone constrained equations. However, the study of conjugate gradient methods for large-scale (un)constrained equations is relatively rare. Cheng [13] proposed a PRP type method (PRPT) for systems of monotone equations, which is a combination of the well-known PRP method and the hyperplane projection method, and the numerical results in [13] show that the PRPT method performs better than the SGP method in [12].
Different from the methods in [7, 8, 10], the methods in [9, 11–13] do not need to solve a linearized equation at each iteration; however, the latter do not investigate the convergent rate, and even we do not know whether they possess the linear convergence rate. In this paper, motivated by the projection methods in [7, 8, 10] and the gradient methods in [9, 12, 13], we propose a matrix-free method for solving nonlinear constrained equations, which can be viewed as a combination of the well-known PRP conjugate gradient method and the famous hyperplane projection method, and it possesses linear convergence rate under standard conditions. The remainder of this paper is organized as follows. Section 2 describes the new method and presents its global convergence analysis. The linear convergence rate of the new method is established in Section 3. Numerical results are reported in Section 4. Finally, some final remarks are included in Section 5.
2. Algorithm and Convergence Analysis
Let denote the solution set of CES (). Throughout this paper, we assume that is nonempty and is monotone; that is, which implies that the solution set is closed. Then let denote the orthogonal projection of a point onto the convex set , which has the following nonexpansive property: Now, we describe the matrix-free method for nonlinear constrained equations.
Algorithm 1.
Consider the following.
Step 0. Given an arbitrary initial point , the parameters , , , and . Given the initial steplength and set .
Step 1. If , then stop; otherwise go to Step 2.
Step 2. Compute by
where
If , set .
Step 3. Find the trial point , where with being the smallest nonnegative integer such that
Step 4. Compute
where
Choose an initial steplength such that . Set and go to Step 1.
Remark 2. Obviously , defined by (4), is motivated by [14], and it is not difficult to deduce that satisfies Therefore, by Cauchy-Schwartz inequality, we have . This together with Step 2 of Algorithm 1 implies
Remark 3. In (7), we attach a relax factor (better when close to 2) to based on numerical experiences.
Remark 4. Line search (6) is different from that of [12, 13], which is well-defined by the following Lemma.
Lemma 5. For all , there exists a nonnegative number satisfying (6).
Proof. In fact, if , then from (10), we have , which means that Algorithm 1 terminates with being a solution of CES (). Now, we consider for all . For the sake of contradiction, we suppose that there exists such that (6) is not satisfied for any nonnegative integer ; that is, Letting and using the continuity of yield On the other hand, by (10), we obtain which together with (12) means that ; however, this contradicts the fact that . Therefore the assertion holds. This completes the proof.
Lemma 6. Suppose that is monotone and let and be the sequences generated by Algorithm 1; then and are both bounded; furthermore, it holds that
Proof. From (6), we have For any , from (3), the nonexpansiveness of the projection operator, it holds that By the monotonicity of mapping , we have Substituting (15) and (17) into (16), we have which together with indicates that, for all , which shows that the sequence is bounded. By (10), it holds that is bounded and so is . Then, by the continuity of , there exists a constant such that for all . Therefore it follows from (18) that which implies that the assertion (14) holds. The proof is completed.
Now, we prove the global convergence of Algorithm 1.
Theorem 7. Suppose that the conditions in Lemma 6 hold. Then the sequence generated by Algorithm 1 globally converges to a solution of CES ().
Proof. We consider the following two possible cases.
Case 1. Consider . Thus, by (10), we have . This together with the continuity of implies that the sequence has some accumulation point such that . From (19), it holds that converges, and since is an accumulation point of , it must hold that converges to .
Case 2. Consider . Then by (14), it follows that . Therefore, from the line search (6), for sufficiently large , we have
Since are both bounded, we can choose a sequence and letting in (21), we can obtain
where and are limit points of corresponding subsequences. On the other hand, by (10), we obtain
Letting in the above inequality, we obtain
Thus, by (22) and (24), we get , and this contradicts the fact that . Therefore, does not hold. This completes the proof.
3. Convergence Rate
By Theorem 7, we know that the sequence generated by Algorithm 1 converges to a solution of CES (). In what follows, we always assume that as , where . To establish the local convergence rate of the sequence generated by Algorithm 1, we need the following assumption.
Assumption 8. For , there exist three positive constants , , and such that where dist denotes the distance from to the solution set , and Now, we analyze the convergence rate of the sequence generated by Algorithm 1 under conditions (25) and (26).
Lemma 9. If the conditions in Assumption 8 hold, then the sequence generated by line search (6) has a positive bound from below.
Proof. We only need to prove that for sufficiently large , has a positive bound from below. If , then by the construction of , we have In addition, by (10), we have Then, by the above two inequalities, we can obtain On the other hand, from (26), we have By (30) and (31), for sufficiently large we obtain Therefore, there is a positive constant , such that for all . The proof is completed.
Theorem 10. In addition to the assumptions in Theorem 7, if conditions (25) and (26) hold, then the sequence Q-linearly converges to 0; hence the whole sequence converges to R-linearly.
Proof. Let be the closest solution to . That is, . By (18), we have For sufficiently large , it follows from (10) and (26) that Thus, from (6), (10), (25), and (33), for sufficiently large , it holds that Substituting the above two inequalities into (34), we have which implies that the sequence -linearly converges to 0. Therefore, the whole sequence converges to -linearly. The proof is completed.
4. Numerical Results
In this section, we test Algorithm 1 and compared it with the projection method in [7] and the spectral gradient projection method in [9]. We give the following two simple problems to test the efficiency of the three methods.
Problem 11. The mapping is taken as , where and . Obviously, this problem has a unique solution .
Problem 12. The mapping is taken as , where
and . Obviously, Problem 12 is nonsmooth at .
The codes are written in Mablab7.0 and run on a personal computer with 2.0 GHZ CPU processor. The parameters used in Algorithm 1 are set as , , , and . The initial steplength in Step 2 of Algorithm 1 is set to be the spectral coefficient
where and . By the monotonicity and the Lipschitz continuity of , it is not difficult to show that
where is the Lipschitz constant. If , we replace the spectral coefficient by
where and . This parabolic model is the same as the one described in [15]. We stop the iteration if the iteration number exceeds 1000 or the inequality is satisfied. The method in [7] (denoted by WPM) is implemented with the following parameters: , , , , and . The method in [9] (denoted by YSGP) is implemented with the following parameters: , , and .
For Problem 11, the initial point is set as , and Table 1 gives the numerical results by Algorithm 1, WPM, and YSGP with different dimensions, where Iter. denotes the iteration number, Fn denotes the number of function evaluations, and CPU denotes the CPU time in seconds when the algorithm terminates. Table 2 lists the numerical results of Problem 12 with different initial points. The numerical results given in Tables 1 and 2 show that Algorithm 1 performs a little better than YSGP in [9] and obviously better than WPM in [7], since it requires much lower number of iterations or less CPU time than WPM in [7] and a little lower number of iterations or less CPU time than YSGP in [9]. So the proposed method is promising.
5. Conclusions
A globally convergent matrix-free method to solve constrained equations has been developed, which is not only derivative-free but also completely matrix-free. Consequently, it can be applied to solve large-scale nonsmooth constrained equations. We established the global convergence without the requirement of differentiability of the equations and presented the linear convergence rate under standard conditions. We also report some numerical results to show the efficiency of the proposed method.
Numerical results indicate that the parameters and influence the performance of the method, so the choice of the positive constants and is our future work.
Conflict of Interests
All the authors of the paper declare that they do not have any conflict of interests, and there are no financial or personal relationships with other people or organizations that can inappropriately influence our work in this paper.
Acknowledgments
The authors would like to thank the referees for giving them many valuable suggestions and comments, which improve this paper greatly. This work is supported by the Nature Science Foundation of Shandong Province (ZR2012AL08).