Abstract and Applied Analysis

Abstract and Applied Analysis / 2013 / Article
Special Issue

Advance in Nonlinear Analysis: Algorithm, Convergence and Applications

View this Special Issue

Research Article | Open Access

Volume 2013 |Article ID 814912 | https://doi.org/10.1155/2013/814912

Yuanying Qiu, Dandan Cui, Wei Xue, Gaohang Yu, "A Self-Adjusting Spectral Conjugate Gradient Method for Large-Scale Unconstrained Optimization", Abstract and Applied Analysis, vol. 2013, Article ID 814912, 8 pages, 2013. https://doi.org/10.1155/2013/814912

A Self-Adjusting Spectral Conjugate Gradient Method for Large-Scale Unconstrained Optimization

Academic Editor: Guoyin Li
Received21 Feb 2013
Accepted17 Mar 2013
Published11 Apr 2013

Abstract

This paper presents a hybrid spectral conjugate gradient method for large-scale unconstrained optimization, which possesses a self-adjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of large-scale problems in CUTEr to show the convergence and efficiency of the proposed method.

1. Introduction

Consider the following unconstrained optimization problem: where is a nonlinear smooth function and its gradient is available. Conjugate gradient methods are very efficient for solving (1), especially when the dimension is large, and have the following iterative form: where is a steplength obtained by a line search, and is the search direction defined by where is a scalar and denotes the gradient of at point .

There are at least six formulas for , which are given below: where and denotes the Euclidean norm. In the above six methods, HS, PR, and LS methods are especially efficient in real computations, but one may not globally converge for general functions. FR, CD, and DY methods are globally convergent, but they perform much worse. To combine the good numerical performance of HS method and the nice global convergence property of DY method, Dai and Yuan [1] proposed an efficient hybrid formula for which is defined as the following form:

Their studies suggested that the HSDY method (5) has the same advantage of avoiding the propensity of short steps as the HS method [1]. They also proved that the HSDY method with the standard wolfe line search produces a descent search direction at each iteration and converges globally. Descent condition may be crucial for the convergence analysis of conjugate gradient methods with inexact line searches [2, 3]. Further, there are some modified conjugate gradient methods [47] which possess the sufficiently descent property without any line search condition. Recently, Yu [8] proposed a spectral version of HSDY method: where with ,  . The numerical experiments show that this simple preconditioning technique benefits to its performance.

In this paper, based on a new conjugate condition [9], we propose a new hybrid spectral conjugate gradient method with defined by where

A full description of DS-HSDY method is formally given as follows.

Algorithm 1 (DS-HSDY conjugate gradient method).
Data. Choose constants , , and . Given an initial point , set . Let .

Step 1. If , then stop.

Step 2. Determine satisfying the standard Wolfe condition: Then update .

Step 3. Compute , and . Then update such as Set and go to Step 1.

The rest of the paper is organized as follows. In the next section, we show that the DS-HSDY method possesses a self-adjusting property. In Section 3, we establish its global convergence result under the standard Wolfe line search conditions. Section 4 gives some numerical results on a set of large-scale unconstrained test problems in CUTEr to illustrate the convergence and efficiency of the proposed method. Finally we have a Conclusion section.

2. Self-Adjusting Property

In this section, we prove that the DS-HSDY method possesses a self-adjusting property. To begin with, we assume that otherwise, a stationary point has been found, and define the two following important quantities: The quantity shows the size of , where is a quantity showing the descent degree of . In fact, if , is a descent direction. Furthermore, if for some constant , then we have the sufficient descent condition On the other hand, it follows from (12) that Hence Combining with (17) yields Dividing both sides of (18) by and using (7), we obtain It follows from (19) and the definitions of and that

Additionally, we assume that there exist positive constants and such that then we have the following result.

Theorem 2. Consider the method (2), (8) and (12), where is a descent direction. If (21) holds, there exist positive constants , , and such that relations hold for all .

Proof. Summing (20) over the iterates and noting that , we get Since , it follows from (25) that Equations (21), (26), and yield Furthermore, we have Thus (24) holds with .
Noting that and , it is easy to derive that (22) and (23) hold with and , respectively. Hence the proof is complete.

Theorem 3. Consider the method (2), (8), and (12), where is a descent direction. If (21) holds, then for any , there exist constants , , and such that, for any k, the relations hold for at least values of .

Proof. The proof is similar to the Theorem  2 in [10], so we omit it here.

Therefore, by Theorems 2 and 3, it was shown that DS-HSDY method possesses a self-adjusting property which is independent of the line search and the function convexity.

3. Global Convergence

Throughout the paper, we assume that the following assumptions hold.

Assumption 1. (1) is bounded below in the level set ;
(2) in a neighborhood of , is differentiable and its gradient is Lipschitz continuous; namely, there exists a constant such that Under Assumption 1 on , we could get a useful lemma.

Lemma 4. Suppose that is a starting point for which Assumption 1 holds. Consider any method in the form (2), where is a descent direction and satisfies the weak Wolfe conditions; then one has that For DS-HSDY method, one has the following global convergence result.

Theorem 5. Suppose that is a starting point for which Assumption 1 hold. Consider DS-HSDY method; if for all , then one has that Further, the method converges in the sense that

Proof. Since , it is obvious that . Assume that . By (10) and the definition of the , we have , then . In addition, from (8), we have Let , then we have . By (12) with replaced by , and multiplying it by , we have From this and the formula for , we get where At the same time, if we define it follows from (39) that Then we have by (10), with replaced by , that Furthermore, we have The above relation, (40), (41), and the fact that imply that . Thus by induction, (32) holds.
We now prove (33) by contradiction and assume that there exists some constant such that Since , we have that Dividing both sides of (44) by and using (36) and (40), we obtain In addition, since and , we have that , or equivalently which with (37) yields By (45) and (47), we obtain Using (48) recursively and noting that , Then we get from this and (43) that which indicates This contradicts the Zoutendijk condition (31). Hence we complete the proof.

4. Numerical Result

In this section, we compare the performance of DS-HSDY method to method [11], HSDY method [1], and S-HSDY method [8]. The test problems are taken from CUTEr (http://hsl.rl.ac.uk/cuter-www/problems.html) with the standard initial points. All codes are written in double precision Fortran and complied with f77 (default compiler settings) on a PC (AMD Athlon XP 2500 + CPU 1.84 GHz). Our line search subroutine computes such that the Wolfe conditions (10) and (11) hold with and . We use the condition or as the stopping criterion. The numerical results are presented in Tables 1, 2, 3, and 4 with the form NI/Nfg/T, where we report the dimension of the problem (), the number of iteration (NI), the number of function evaluations (Nfg), and the CPU time () in 0.01 seconds.


Function NI Nfg (0.01 S)

Quadratic QF2 10000 2227 2885 2016
Extended EP1 10000 4 7 3
Extended Tridiagonal 2 10000 39 98 47
ARGLINA 10000 5 15 4
ARWHEAD 10000 7 14 21
BDQRTIC 5000 157 720 526
BDEXP 5000 6 8 7
BRYBND 5000 5 11 1215
COSINE 10000 21 45 39
CRAGGLVY 10000 129 250 444
DIXMAANA 10000 6 12 19
DIXMAANB 10000 8 16 26
DIXMAANC 10000 11 23 38
DIXMAAND 10000 13 29 44
DIXMAANE 5000 558 799 712
DIXMAANF 5000 558 598 525
DIXMAANG 5000 519 784 684
DIXMAANH 5000 379 3488 2469
DIXMAANI 5000 593 854 755
DIXMAANJ 5000 492 751 651
DIXMAANK 5000 653 979 863
DQDRTIC 10000 11 23 19
DQRTIC 10000 33 57 42
EDENSCH 10000 26 90 78
EG2 10000 209 1426 473
ENGVAL1 10000 30 93 21
EXTROSNB 10000 29 63 25
FREUROTH 10000 61 145 81
LIARWHD 10000 25 49 36
NONDIA 10000 9 17 17
NONDQUAR 5000 1786 3258 1752
NONSCOMP 10000 5001 6799 9751


Function NI Nfg (0.01 S)

Quadratic QF2 10000 1593 1902 1876
Extended EP1 10000 4 7 4
Extended Tridiagonal 2 10000 34 55 32
ARGLINA 10000 5 15 4
ARWHEAD 10000 13 58 71
BDQRTIC 5000 171 567 422
BDEXP 5000 6 8 6
BRYBND 5000 5 11 1222
COSINE 10000 21 46 41
CRAGGLVY 10000 109 255 434
DIXMAANA 10000 5 10 16
DIXMAANB 10000 9 18 29
DIXMAANC 10000 10 21 33
DIXMAAND 10000 13 29 45
DIXMAANE 5000 446 541 493
DIXMAANF 5000 389 876 690
DIXMAANG 5000 552 660 602
DIXMAANH 5000 202 5106 3417
DIXMAANI 5000 365 450 409
DIXMAANJ 5000 444 532 484
DIXMAANK 5000 367 452 410
DQDRTIC 10000 8 17 14
DQRTIC 10000 37 68 48
EDENSCH 10000 30 99 85
EG2 10000 305 2811 879
ENGVAL1 10000 30 52 21
EXTROSNB 10000 27 54 21
FREUROTH 10000 143 283 177
LIARWHD 10000 32 62 44
NONDIA 10000 7 14 14
NONDQUAR 5000 2049 3730 2011
NONSCOMP 10000 58 100 98


Function NI Nfg (0.01 S)

Quadratic QF2 10000 1582 1941 1836
Extended EP1 10000 4 7 3
Extended Tridiagonal 2 10000 34 55 34
ARGLINA 10000 5 15 3
ARWHEAD 10000 13 58 75
BDQRTIC 5000 111 526 377
BDEXP 5000 6 8 5
BRYBND 5000 5 11 1179
COSINE 10000 21 46 39
CRAGGLVY 10000 103 189 332
DIXMAANA 10000 5 10 15
DIXMAANB 10000 9 18 30
DIXMAANC 10000 10 21 33
DIXMAAND 10000 13 29 43
DIXMAANE 5000 422 514 468
DIXMAANF 5000 310 792 618
DIXMAANG 5000 410 495 449
DIXMAANH 5000 217 6957 4642
DIXMAANI 5000 380 450 417
DIXMAANJ 5000 359 438 402
DIXMAANK 5000 404 485 448
DQDRTIC 10000 8 17 14
DQRTIC 10000 37 68 49
EDENSCH 10000 30 99 84
EG2 10000 242 1731 570
ENGVAL1 10000 29 124 22
EXTROSNB 10000 27 54 22
FREUROTH 10000 214 408 260
LIARWHD 10000 27 54 37
NONDIA 10000 7 14 14
NONDQUAR 5000 1782 3210 1738
NONSCOMP 10000 58 100 100


Function NI Nfg (0.01 S)

Quadratic QF2 10000 1623 1978 1783
Extended EP1 10000 4 7 3
Extended Tridiagonal 2 10000 34 55 30
ARGLINA 10000 5 15 3
ARWHEAD 10000 13 58 70
BDQRTIC 5000 165 448 324
BDEXP 5000 6 8 4
BRYBND 5000 5 11 990
COSINE 10000 14 38 28
CRAGGLVY 10000 110 150 266
DIXMAANA 10000 5 10 16
DIXMAANB 10000 9 18 27
DIXMAANC 10000 10 21 31
DIXMAAND 10000 13 29 44
DIXMAANE 5000 410 493 430
DIXMAANF 5000 432 546 469
DIXMAANG 5000 476 582 505
DIXMAANH 5000 442 1204 7792
DIXMAANI 5000 397 467 408
DIXMAANJ 5000 445 594 503
DIXMAANK 5000 403 507 438
DQDRTIC 10000 10 21 17
DQRTIC 10000 35 62 43
EDENSCH 10000 29 87 70
EG2 10000 251 1121 381
ENGVAL1 10000 29 50 19
EXTROSNB 10000 65 122 44
FREUROTH 10000 50 133 67
LIARWHD 10000 47 94 61
NONDIA 10000 7 14 12
NONDQUAR 5000 1831 3262 1665
NONSCOMP 10000 73 126 119

Figure 1 shows the performance of these test methods relative to the CPU time, which were evaluated using the profiles of Dolan and Moré [12]. That is, for each method, we plot the fraction of problems for which the method is within a factor of the best time. The top curve is the method that solved the most problems in a time that was within a factor of the best time. Clearly, the left side of the figure gives the percentage of the test problems for which a method is the fastest. As we can see from Figure 1, DS-HSDY method has the best performance which performs better than S-HSDY method, HSDY method, and the well-known method.

5. Conclusion

In this paper, we proposed an efficient hybrid spectral conjugate gradient method with self-adjusting property. Under some suitable assumptions, we established the global convergence result for the DS-HSDY method. Numerical results indicated that the proposed method is efficient for large-scale unconstrained optimization problems.

Acknowledgments

This work was partly supported by the National Natural Science Foundation of China (no. 61262026), the JGZX program of Jiangxi Province (20112BCB23027), and the science and technology program of Jiangxi Education Committee (LDJH12088). The authors would also like to thank the editor and an anonymous referees for their comments and suggestions on the first version of the paper, which led to significant improvements of the presentation.

References

  1. Y. H. Dai and Y. Yuan, “An efficient hybrid conjugate gradient method for unconstrained optimization,” Annals of Operations Research, vol. 103, pp. 33–47, 2001. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  2. Y. H. Dai and Y. Yuan, “A nonlinear conjugate gradient method with a strong global convergence property,” SIAM Journal on Optimization, vol. 10, no. 1, pp. 177–182, 1999. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  3. Z. X. Wei, G. Y. Li, and L. Q. Qi, “Global convergence of the Polak-Ribière-Polyak conjugate gradient method with an Armijo-type inexact line search for nonconvex unconstrained optimization problems,” Mathematics of Computation, vol. 77, no. 264, pp. 2173–2193, 2008. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  4. W. W. Hager and H. Zhang, “A new conjugate gradient method with guaranteed descent and an efficient line search,” SIAM Journal on Optimization, vol. 16, no. 1, pp. 170–192, 2005. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  5. L. Zhang, W. Zhou, and D.-H. Li, “A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence,” IMA Journal of Numerical Analysis, vol. 26, no. 4, pp. 629–640, 2006. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  6. G. Yuan, “Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems,” Optimization Letters, vol. 3, no. 1, pp. 11–21, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  7. G. Yu, L. Guan, and W. Chen, “Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization,” Optimization Methods and Software, vol. 23, no. 2, pp. 275–293, 2008. View at: Publisher Site | Google Scholar | MathSciNet
  8. G. Yu, Nonlinear self-scaling conjugate gradient methods for large-scale optimization problems [Ph.D. thesis], Sun Yat-Sen University, Guangzhou, China, 2007.
  9. G. Li, C. Tang, and Z. Wei, “New conjugacy condition and related new conjugate gradient methods for unconstrained optimization,” Journal of Computational and Applied Mathematics, vol. 202, no. 2, pp. 523–539, 2007. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  10. Y.-H. Dai, “New properties of a nonlinear conjugate gradient method,” Numerische Mathematik, vol. 89, no. 1, pp. 83–98, 2001. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  11. J. C. Gilbert and J. Nocedal, “Global convergence properties of conjugate gradient methods for optimization,” SIAM Journal on Optimization, vol. 2, no. 1, pp. 21–42, 1992. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet
  12. E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming, vol. 91, no. 2, pp. 201–213, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH | MathSciNet

Copyright © 2013 Yuanying Qiu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views1012
Downloads775
Citations

Related articles

We are committed to sharing findings related to COVID-19 as quickly as possible. We will be providing unlimited waivers of publication charges for accepted research articles as well as case reports and case series related to COVID-19. Review articles are excluded from this waiver policy. Sign up here as a reviewer to help fast-track new submissions.