Advance in Nonlinear Analysis: Algorithm, Convergence and Applications
View this Special IssueResearch Article  Open Access
Yuanying Qiu, Dandan Cui, Wei Xue, Gaohang Yu, "A SelfAdjusting Spectral Conjugate Gradient Method for LargeScale Unconstrained Optimization", Abstract and Applied Analysis, vol. 2013, Article ID 814912, 8 pages, 2013. https://doi.org/10.1155/2013/814912
A SelfAdjusting Spectral Conjugate Gradient Method for LargeScale Unconstrained Optimization
Abstract
This paper presents a hybrid spectral conjugate gradient method for largescale unconstrained optimization, which possesses a selfadjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of largescale problems in CUTEr to show the convergence and efficiency of the proposed method.
1. Introduction
Consider the following unconstrained optimization problem: where is a nonlinear smooth function and its gradient is available. Conjugate gradient methods are very efficient for solving (1), especially when the dimension is large, and have the following iterative form: where is a steplength obtained by a line search, and is the search direction defined by where is a scalar and denotes the gradient of at point .
There are at least six formulas for , which are given below: where and denotes the Euclidean norm. In the above six methods, HS, PR, and LS methods are especially efficient in real computations, but one may not globally converge for general functions. FR, CD, and DY methods are globally convergent, but they perform much worse. To combine the good numerical performance of HS method and the nice global convergence property of DY method, Dai and Yuan [1] proposed an efficient hybrid formula for which is defined as the following form:
Their studies suggested that the HSDY method (5) has the same advantage of avoiding the propensity of short steps as the HS method [1]. They also proved that the HSDY method with the standard wolfe line search produces a descent search direction at each iteration and converges globally. Descent condition may be crucial for the convergence analysis of conjugate gradient methods with inexact line searches [2, 3]. Further, there are some modified conjugate gradient methods [4ā7] which possess the sufficiently descent property without any line search condition. Recently, Yu [8] proposed a spectral version of HSDY method: where with ,āā. The numerical experiments show that this simple preconditioning technique benefits to its performance.
In this paper, based on a new conjugate condition [9], we propose a new hybrid spectral conjugate gradient method with defined by where
A full description of DSHSDY method is formally given as follows.
Algorithm 1 (DSHSDY conjugate gradient method).
Data. Choose constants , , and . Given an initial point , set . Let .
Step 1. If , then stop.
Step 2. Determine satisfying the standard Wolfe condition: Then update .
Step 3. Compute , and . Then update such as Set and go to Step 1.
The rest of the paper is organized as follows. In the next section, we show that the DSHSDY method possesses a selfadjusting property. In Section 3, we establish its global convergence result under the standard Wolfe line search conditions. Section 4 gives some numerical results on a set of largescale unconstrained test problems in CUTEr to illustrate the convergence and efficiency of the proposed method. Finally we have a Conclusion section.
2. SelfAdjusting Property
In this section, we prove that the DSHSDY method possesses a selfadjusting property. To begin with, we assume that otherwise, a stationary point has been found, and define the two following important quantities: The quantity shows the size of , where is a quantity showing the descent degree of . In fact, if , is a descent direction. Furthermore, if for some constant , then we have the sufficient descent condition On the other hand, it follows from (12) that Hence Combining with (17) yields Dividing both sides of (18) by and using (7), we obtain It follows from (19) and the definitions of and that
Additionally, we assume that there exist positive constants and such that then we have the following result.
Theorem 2. Consider the method (2), (8) and (12), where is a descent direction. If (21) holds, there exist positive constants , , and such that relations hold for all .
Proof. Summing (20) over the iterates and noting that , we get
Since , it follows from (25) that
Equations (21), (26), and yield
Furthermore, we have
Thus (24) holds with .
Noting that and , it is easy to derive that (22) and (23) hold with and , respectively. Hence the proof is complete.
Theorem 3. Consider the method (2), (8), and (12), where is a descent direction. If (21) holds, then for any , there exist constants , , and such that, for any k, the relations hold for at least values of .
Proof. The proof is similar to the Theoremāā2 in [10], so we omit it here.
Therefore, by Theorems 2 and 3, it was shown that DSHSDY method possesses a selfadjusting property which is independent of the line search and the function convexity.
3. Global Convergence
Throughout the paper, we assume that the following assumptions hold.
Assumption 1. (1)āis bounded below in the level set ;
(2) in a neighborhood of , is differentiable and its gradient is Lipschitz continuous; namely, there exists a constant such that
Under Assumption 1 on , we could get a useful lemma.
Lemma 4. Suppose that is a starting point for which Assumption 1 holds. Consider any method in the form (2), where is a descent direction and satisfies the weak Wolfe conditions; then one has that For DSHSDY method, one has the following global convergence result.
Theorem 5. Suppose that is a starting point for which Assumption 1 hold. Consider DSHSDY method; if for all , then one has that Further, the method converges in the sense that
Proof. Since , it is obvious that . Assume that . By (10) and the definition of the , we have , then . In addition, from (8), we have
Let , then we have . By (12) with replaced by , and multiplying it by , we have
From this and the formula for , we get
where
At the same time, if we define
it follows from (39) that
Then we have by (10), with replaced by , that
Furthermore, we have
The above relation, (40), (41), and the fact that imply that . Thus by induction, (32) holds.
We now prove (33) by contradiction and assume that there exists some constant such that
Since , we have that
Dividing both sides of (44) by and using (36) and (40), we obtain
In addition, since and , we have that , or equivalently
which with (37) yields
By (45) and (47), we obtain
Using (48) recursively and noting that ,
Then we get from this and (43) that
which indicates
This contradicts the Zoutendijk condition (31). Hence we complete the proof.
4. Numerical Result
In this section, we compare the performance of DSHSDY method to method [11], HSDY method [1], and SHSDY method [8]. The test problems are taken from CUTEr (http://hsl.rl.ac.uk/cuterwww/problems.html) with the standard initial points. All codes are written in double precision Fortran and complied with f77 (default compiler settings) on a PC (AMD Athlon XP 2500 + CPU 1.84āGHz). Our line search subroutine computes such that the Wolfe conditions (10) and (11) hold with and . We use the condition or as the stopping criterion. The numerical results are presented in Tables 1, 2, 3, and 4 with the form NI/Nfg/T, where we report the dimension of the problem (), the number of iteration (NI), the number of function evaluations (Nfg), and the CPU time () in 0.01 seconds.




Figure 1 shows the performance of these test methods relative to the CPU time, which were evaluated using the profiles of Dolan and MorĆ© [12]. That is, for each method, we plot the fraction of problems for which the method is within a factor of the best time. The top curve is the method that solved the most problems in a time that was within a factor of the best time. Clearly, the left side of the figure gives the percentage of the test problems for which a method is the fastest. As we can see from Figure 1, DSHSDY method has the best performance which performs better than SHSDY method, HSDY method, and the wellknown method.
5. Conclusion
In this paper, we proposed an efficient hybrid spectral conjugate gradient method with selfadjusting property. Under some suitable assumptions, we established the global convergence result for the DSHSDY method. Numerical results indicated that the proposed method is efficient for largescale unconstrained optimization problems.
Acknowledgments
This work was partly supported by the National Natural Science Foundation of China (no. 61262026), the JGZX program of Jiangxi Province (20112BCB23027), and the science and technology program of Jiangxi Education Committee (LDJH12088). The authors would also like to thank the editor and an anonymous referees for their comments and suggestions on the first version of the paper, which led to significant improvements of the presentation.
References
 Y. H. Dai and Y. Yuan, āAn efficient hybrid conjugate gradient method for unconstrained optimization,ā Annals of Operations Research, vol. 103, pp. 33ā47, 2001. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Y. H. Dai and Y. Yuan, āA nonlinear conjugate gradient method with a strong global convergence property,ā SIAM Journal on Optimization, vol. 10, no. 1, pp. 177ā182, 1999. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Z. X. Wei, G. Y. Li, and L. Q. Qi, āGlobal convergence of the PolakRibièrePolyak conjugate gradient method with an Armijotype inexact line search for nonconvex unconstrained optimization problems,ā Mathematics of Computation, vol. 77, no. 264, pp. 2173ā2193, 2008. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 W. W. Hager and H. Zhang, āA new conjugate gradient method with guaranteed descent and an efficient line search,ā SIAM Journal on Optimization, vol. 16, no. 1, pp. 170ā192, 2005. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 L. Zhang, W. Zhou, and D.H. Li, āA descent modified PolakRibièrePolyak conjugate gradient method and its global convergence,ā IMA Journal of Numerical Analysis, vol. 26, no. 4, pp. 629ā640, 2006. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 G. Yuan, āModified nonlinear conjugate gradient methods with sufficient descent property for largescale optimization problems,ā Optimization Letters, vol. 3, no. 1, pp. 11ā21, 2009. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 G. Yu, L. Guan, and W. Chen, āSpectral conjugate gradient methods with sufficient descent property for largescale unconstrained optimization,ā Optimization Methods and Software, vol. 23, no. 2, pp. 275ā293, 2008. View at: Publisher Site  Google Scholar  MathSciNet
 G. Yu, Nonlinear selfscaling conjugate gradient methods for largescale optimization problems [Ph.D. thesis], Sun YatSen University, Guangzhou, China, 2007.
 G. Li, C. Tang, and Z. Wei, āNew conjugacy condition and related new conjugate gradient methods for unconstrained optimization,ā Journal of Computational and Applied Mathematics, vol. 202, no. 2, pp. 523ā539, 2007. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 Y.H. Dai, āNew properties of a nonlinear conjugate gradient method,ā Numerische Mathematik, vol. 89, no. 1, pp. 83ā98, 2001. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 J. C. Gilbert and J. Nocedal, āGlobal convergence properties of conjugate gradient methods for optimization,ā SIAM Journal on Optimization, vol. 2, no. 1, pp. 21ā42, 1992. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 E. D. Dolan and J. J. Moré, āBenchmarking optimization software with performance profiles,ā Mathematical Programming, vol. 91, no. 2, pp. 201ā213, 2002. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
Copyright
Copyright © 2013 Yuanying Qiu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.