Abstract

This paper presents a hybrid spectral conjugate gradient method for large-scale unconstrained optimization, which possesses a self-adjusting property. Under the standard Wolfe conditions, its global convergence result is established. Preliminary numerical results are reported on a set of large-scale problems in CUTEr to show the convergence and efficiency of the proposed method.

1. Introduction

Consider the following unconstrained optimization problem: where is a nonlinear smooth function and its gradient is available. Conjugate gradient methods are very efficient for solving (1), especially when the dimension is large, and have the following iterative form: where is a steplength obtained by a line search, and is the search direction defined by where is a scalar and denotes the gradient of at point .

There are at least six formulas for , which are given below: where and denotes the Euclidean norm. In the above six methods, HS, PR, and LS methods are especially efficient in real computations, but one may not globally converge for general functions. FR, CD, and DY methods are globally convergent, but they perform much worse. To combine the good numerical performance of HS method and the nice global convergence property of DY method, Dai and Yuan [1] proposed an efficient hybrid formula for which is defined as the following form:

Their studies suggested that the HSDY method (5) has the same advantage of avoiding the propensity of short steps as the HS method [1]. They also proved that the HSDY method with the standard wolfe line search produces a descent search direction at each iteration and converges globally. Descent condition may be crucial for the convergence analysis of conjugate gradient methods with inexact line searches [2, 3]. Further, there are some modified conjugate gradient methods [47] which possess the sufficiently descent property without any line search condition. Recently, Yu [8] proposed a spectral version of HSDY method: where with ,  . The numerical experiments show that this simple preconditioning technique benefits to its performance.

In this paper, based on a new conjugate condition [9], we propose a new hybrid spectral conjugate gradient method with defined by where

A full description of DS-HSDY method is formally given as follows.

Algorithm 1 (DS-HSDY conjugate gradient method).
Data. Choose constants , , and . Given an initial point , set . Let .

Step 1. If , then stop.

Step 2. Determine satisfying the standard Wolfe condition: Then update .

Step 3. Compute , and . Then update such as Set and go to Step 1.

The rest of the paper is organized as follows. In the next section, we show that the DS-HSDY method possesses a self-adjusting property. In Section 3, we establish its global convergence result under the standard Wolfe line search conditions. Section 4 gives some numerical results on a set of large-scale unconstrained test problems in CUTEr to illustrate the convergence and efficiency of the proposed method. Finally we have a Conclusion section.

2. Self-Adjusting Property

In this section, we prove that the DS-HSDY method possesses a self-adjusting property. To begin with, we assume that otherwise, a stationary point has been found, and define the two following important quantities: The quantity shows the size of , where is a quantity showing the descent degree of . In fact, if , is a descent direction. Furthermore, if for some constant , then we have the sufficient descent condition On the other hand, it follows from (12) that Hence Combining with (17) yields Dividing both sides of (18) by and using (7), we obtain It follows from (19) and the definitions of and that

Additionally, we assume that there exist positive constants and such that then we have the following result.

Theorem 2. Consider the method (2), (8) and (12), where is a descent direction. If (21) holds, there exist positive constants , , and such that relations hold for all .

Proof. Summing (20) over the iterates and noting that , we get Since , it follows from (25) that Equations (21), (26), and yield Furthermore, we have Thus (24) holds with .
Noting that and , it is easy to derive that (22) and (23) hold with and , respectively. Hence the proof is complete.

Theorem 3. Consider the method (2), (8), and (12), where is a descent direction. If (21) holds, then for any , there exist constants , , and such that, for any k, the relations hold for at least values of .

Proof. The proof is similar to the Theorem  2 in [10], so we omit it here.

Therefore, by Theorems 2 and 3, it was shown that DS-HSDY method possesses a self-adjusting property which is independent of the line search and the function convexity.

3. Global Convergence

Throughout the paper, we assume that the following assumptions hold.

Assumption 1. (1) is bounded below in the level set ;
(2) in a neighborhood of , is differentiable and its gradient is Lipschitz continuous; namely, there exists a constant such that Under Assumption 1 on , we could get a useful lemma.

Lemma 4. Suppose that is a starting point for which Assumption 1 holds. Consider any method in the form (2), where is a descent direction and satisfies the weak Wolfe conditions; then one has that For DS-HSDY method, one has the following global convergence result.

Theorem 5. Suppose that is a starting point for which Assumption 1 hold. Consider DS-HSDY method; if for all , then one has that Further, the method converges in the sense that

Proof. Since , it is obvious that . Assume that . By (10) and the definition of the , we have , then . In addition, from (8), we have Let , then we have . By (12) with replaced by , and multiplying it by , we have From this and the formula for , we get where At the same time, if we define it follows from (39) that Then we have by (10), with replaced by , that Furthermore, we have The above relation, (40), (41), and the fact that imply that . Thus by induction, (32) holds.
We now prove (33) by contradiction and assume that there exists some constant such that Since , we have that Dividing both sides of (44) by and using (36) and (40), we obtain In addition, since and , we have that , or equivalently which with (37) yields By (45) and (47), we obtain Using (48) recursively and noting that , Then we get from this and (43) that which indicates This contradicts the Zoutendijk condition (31). Hence we complete the proof.

4. Numerical Result

In this section, we compare the performance of DS-HSDY method to method [11], HSDY method [1], and S-HSDY method [8]. The test problems are taken from CUTEr (http://hsl.rl.ac.uk/cuter-www/problems.html) with the standard initial points. All codes are written in double precision Fortran and complied with f77 (default compiler settings) on a PC (AMD Athlon XP 2500 + CPU 1.84 GHz). Our line search subroutine computes such that the Wolfe conditions (10) and (11) hold with and . We use the condition or as the stopping criterion. The numerical results are presented in Tables 1, 2, 3, and 4 with the form NI/Nfg/T, where we report the dimension of the problem (), the number of iteration (NI), the number of function evaluations (Nfg), and the CPU time () in 0.01 seconds.

Figure 1 shows the performance of these test methods relative to the CPU time, which were evaluated using the profiles of Dolan and Moré [12]. That is, for each method, we plot the fraction of problems for which the method is within a factor of the best time. The top curve is the method that solved the most problems in a time that was within a factor of the best time. Clearly, the left side of the figure gives the percentage of the test problems for which a method is the fastest. As we can see from Figure 1, DS-HSDY method has the best performance which performs better than S-HSDY method, HSDY method, and the well-known method.

5. Conclusion

In this paper, we proposed an efficient hybrid spectral conjugate gradient method with self-adjusting property. Under some suitable assumptions, we established the global convergence result for the DS-HSDY method. Numerical results indicated that the proposed method is efficient for large-scale unconstrained optimization problems.

Acknowledgments

This work was partly supported by the National Natural Science Foundation of China (no. 61262026), the JGZX program of Jiangxi Province (20112BCB23027), and the science and technology program of Jiangxi Education Committee (LDJH12088). The authors would also like to thank the editor and an anonymous referees for their comments and suggestions on the first version of the paper, which led to significant improvements of the presentation.