A Globally Convergent Hybrid Conjugate Gradient Method and Its Numerical Behaviors

Abstract

We consider a hybrid Dai-Yuan conjugate gradient method. We confirm that its numerical performance can be improved provided that this method uses a practical steplength rule developed by Dong, and the associated convergence is analyzed as well.

1. Introduction

Consider the following problem of finding an 𝑥∈𝑅^𝑛 such that 𝑔(𝑥)=0, (1) where 𝑅^𝑛 is the 𝑛-dimensional Euclidean space and 𝑔∶𝑅^𝑛→𝑅^𝑛 is continuous. Throughout this paper, this problem corresponds to optimality condition of a certain problem of minimizing 𝑓∶𝑅^𝑛→𝑅 which may be not easy to calculate or cannot be expressed as elementary functions. When the dimension 𝑛 is large, conjugate gradient methods can be efficient to solve problem (1). For any given starting point 𝑥₀∈𝑅^𝑛, a sequence {𝑥_𝑘} is generated by the following recursive relation: 𝑥_𝑘+1=𝑥_𝑘+𝛼_𝑘𝑑_𝑘, (2) with 𝑑_𝑘=−𝑔_𝑘,if𝑘=0;−𝑔_𝑘+𝛽_𝑘𝑑_𝑘−1,if𝑘≥1, (3) where 𝛼_𝑘 is a steplength, 𝑑_𝑘 is a descent direction, 𝑔_𝑘 stands for 𝑔(𝑥_𝑘), and 𝛽_𝑘 is a parameter. Different choices of 𝛽_𝑘 result in different nonlinear conjugate gradient methods. The Dai-Yuan (DY) formula [1] and the Hestenes-Stiefel (HS) formula [2] are two famous ones, and are given by 𝛽^DY_𝑘=𝑔_𝑘²𝑑^𝑇_𝑘−1𝑦_𝑘−1,𝛽^HS_𝑘=𝑔^𝑇_𝑘𝑦_𝑘−1𝑑^𝑇_𝑘−1𝑦_𝑘−1, (4) respectively,  where  ‖⋅‖  means  the  2-norm  and  𝑦_𝑘−1=𝑔_𝑘−𝑔_𝑘−1.  Other  well-known  formulae for 𝛽_𝑘 such as the Fletcher-Reeves formula [3], the Polak-Ribière-Polyak formula [4, 5], and the Hager-Zhang formula [6], please refer to [7, 8] for further survey.

In [1], the steplength 𝛼_𝑘 is obtained by the following weak Wolfe line search: 𝑓𝑥_𝑘+𝛼_𝑘𝑑_𝑘≤𝑓𝑥_𝑘+𝛿𝛼_𝑘𝑔^𝑇_𝑘𝑑_𝑘;𝜎𝑔^𝑇_𝑘𝑑_𝑘≤𝑔𝑥_𝑘+𝛼_𝑘𝑑_𝑘^𝑇𝑑_𝑘, (5)(6) where 0<𝛿<𝜎<1. With the same line search, two hybrid versions related to the DY method and the HS method were proposed in [9], which generate the parameter 𝛽_𝑘 by 𝛽^DYHS_𝑘=max−1−𝜎𝛽1+𝜎^DY𝑘,min𝛽^DY_𝑘,𝛽^HS_𝑘,𝛽^DYHS+_𝑘=max0,min𝛽^DY_𝑘,𝛽^HS_𝑘, (7) respectively. And initial numerical results in [10] suggested that the two hybrid conjugate gradient methods (abbreviated as DYHS and DYHS+, resp.) are efficient, especially the DYHS+ method performed better.

The line search plays an important role in the efficiency of conjugate gradient methods. Hager and Zhang [6] showed that the first condition (5) of the weak Wolfe line search limits the accuracy of a conjugate gradient method to the order of the square root of the machine precision (see also [11, 12]); thus, in order to get higher precision, they proposed approximate Wolfe conditions [6, 7], 𝜎𝑔^𝑇_𝑘𝑑_𝑘≤𝑔𝑥_𝑘+𝛼_𝑘𝑑_𝑘^𝑇𝑑_𝑘≤(2𝛿−1)𝑔^𝑇_𝑘𝑑_𝑘, (8) where 0<𝛿<1/2 and 𝛿<𝜎<1, which are usually used combining with the weak Wolfe line search. However, there is no theory to guarantee convergence in [6–8]. By following a referee’s suggestion, we adapt the approximate Wolfe conditions to a Dai-Yuan hybrid conjugate gradient method and investigate its numerical performances.

More recently, Dong designed a practical Armijo-type steplength rule [13] only using gradient, please see [14] for a more conceptual version, the steplength 𝛼_𝑘 is chosen by the following steps: choose 0<𝑡<1, compute some appropriate initial steplength 𝜌_𝑘>0, determine a real number 𝜇_𝑘 (see (12)) and find 𝛼_𝑘 to be the largest 𝛼∈{𝜌_𝑘𝑡^𝑗∣𝑗=0,1,2,…} such that 𝑔𝑥_𝑘+𝛼𝑑_𝑘^𝑇𝑑_𝑘−12𝜇_𝑘𝛼𝑑_𝑘²≤𝜎𝑔^𝑇_𝑘𝑑_𝑘𝑔𝑥,𝑘≤𝐾;_𝑘+𝛼𝑑_𝑘^𝑇𝑑_𝑘+12max−𝜇_𝑘,0𝛼𝑑_𝑘²≤𝜎𝑔^𝑇_𝑘𝑑_𝑘,𝑘>𝐾, (9) where 𝐾 is a nonnegative integer and 0<𝜎<1.

The main differences between the practical steplength rule and the weak Wolfe conditions are that the former does not require function evaluations, it is in high accuracy and has broader application scope. The feature of high accuracy is supported by the corresponding theory analysis in [6, 11]. Numerical results reported in [13] also imply that the line search (9) is efficient and highly accurate. So, it is meaningful to imbed the line search into the hybrid conjugate gradient method with parameter 𝛽^DYHS+_𝑘 and to check its efficiency.

This paper is to solve problem (1), which corresponds to optimality condition of a certain problem of minimizing 𝑓∶𝑅^𝑛→𝑅. If the original function 𝑓 cannot be expressed as elementary functions, the weak Wolfe conditions cannot be applied directly, while the practical steplength rule (9) and the approximate Wolfe conditions (8) can be used to solve this kind of nonlinear unconstrained minimization problems. So, in order to investigate the numerical performances of the two modified methods with steplength rules (9) and (8) and to confirm their broader application scope, two classes of test problems are selected. One class is composed of unconstrained nonlinear optimization problems from the CUTEr library, and the other class is composed of some boundary value problems.

The rest of this paper is organized as follows. In Section 2, we give some basic definitions and properties used in this paper. In Section 3, we describe two modified versions of the Dai-Yuan hybrid conjugate gradient method with the line search (9) and the approximate Wolfe conditions (8) and illustrate that if 𝑔 is Lipschitz continuous, the former version is convergent in the sense that liminf_𝑘→+∞‖𝑔_𝑘‖=0. In Section 4, we test the modified hybrid conjugate gradient methods over two classes of standard test problems and compare them with the DYHS method and the DYHS+ method. Finally, some conclusions are given in Section 5.

2. Preliminaries

In this section, we give some basic definitions and related properties which will be used in the following discussions.

Assumption 1. Assume 𝑔∶𝑅^𝑛→𝑅^𝑛 is 𝐿-Lipschitz continuous in 𝑋⊂𝑅^𝑛, that is, there exists a constant 𝐿>0 such that 𝑔(𝑥)−𝑔𝑦≤𝐿𝑥−𝑦,∀𝑥,𝑦∈𝑋. (10) And its original function 𝑓∶𝑅^𝑛→𝑅 is bounded below in the level set ℒ={𝑥∈𝑅^𝑛∶𝑓(𝑥)≤𝑓(𝑥₀)}.

Definition 2. A mapping 𝑔∶𝑅^𝑛→𝑅^𝑛 is said to be 𝜇-monotone on the set 𝑋⊂𝑅^𝑛 if there exists 𝜇∈𝑅 such that 𝑔(𝑥)−𝑔𝑦,𝑥−𝑦≥𝜇𝑥−𝑦²,∀𝑥,𝑦∈𝑋. (11)

By using the definition above, we know that if the gradient 𝑔∶𝑅^𝑛→𝑅^𝑛 is 𝐿-Lipschitz continuous, then for any given 𝑥_𝑘,𝑑_𝑘∈𝑅^𝑛, the gradient 𝑔 must be 𝜇_𝑘-monotone along the ray {𝑥_𝑘+𝛼𝑑_𝑘∶𝛼≥0} for some 𝜇_𝑘∈[−𝐿,𝐿]. Then, how to evaluate such 𝜇_𝑘? In [13], it is suggested to evaluate using the following approximation formula: 𝜇_𝑘≈𝑔𝑥_𝑘+𝛼_𝑘−1𝑑_𝑘−𝑔_𝑘^𝑇𝑑_𝑘𝛼_𝑘−1𝑑_𝑘². (12)

Lemma 3. For any given 𝑥,𝑑∈𝑅^𝑛 and for all 𝛼≥0, if 𝑓∶𝑅^𝑛→𝑅 is continuously differentiable and the given gradient 𝑔∶𝑅^𝑛→𝑅^𝑛 is 𝜇-monotone along the ray {𝑥+𝛼𝑑∶𝛼≥0}, then the following inequality holds: 𝑓(𝑥+𝛼𝑑)≤𝑓(𝑥)+𝛼𝑔(𝑥+𝛼𝑑)^𝑇1𝑑−2𝜇𝛼²‖𝑑‖². (13)

Proof. Please see [13] for a detailed proof.

3. Algorithm and Its Convergence

In this section, we first formally describe the hybrid conjugate gradient method. Then, we give its two modified versions and illustrate that the modified version with steplength rule (9) is convergent.

Algorithm 4. Step 0. Choose 𝜖>0. Set 𝑑₀=−𝑔₀,𝛼₋₁=1 and 𝑘∶=0.

Step 1. If ‖𝑔_𝑘‖_∞≤𝜖, then stop, otherwise find 𝛼_𝑘 such that certain steplength rule holds.

Step 2. Compute the new iterate by (2) and the new search direction by 𝑑_𝑘+1=−𝑔_𝑘+1+𝛽^DYHS+_𝑘+1𝑑_𝑘. (14)

Set 𝑘∶=𝑘+1 and go to Step 1.

Denote 𝜌_𝑘=max10⁻⁹,1max10⁻⁹,𝜇_𝑘min10⁹,−𝑔^𝑇_𝑘𝑑_𝑘𝑔_𝑘²𝑔_𝑘²𝑑_𝑘². (15) (i)We abbreviate Algorithm 4 as MDYHS+, if the steplength rule in Step 1 is finding 𝛼_𝑘 to be the largest 𝛼∈{𝜌_𝑘𝑡^𝑗∣𝑗=0,1,2,…} (0<𝑡<1) such that 𝑔𝑥_𝑘+𝛼𝑑_𝑘^𝑇𝑑_𝑘+12max−𝜇_𝑘,0𝛼𝑑_𝑘²≤𝜎𝑔^𝑇_𝑘𝑑_𝑘 (16) holds, where 𝜇_𝑘 is defined in (12) and 0<𝜎<1. (ii)And we abbreviate Algorithm 4 as MDYHS + 1, if 𝛼_𝑘 in Step 1 is located to satisfy the approximate Wolfe conditions (8). It should be noticed that on the face of it, the approximate Wolfe conditions only use gradient information to locate the steplength, while they require function evaluations in practical implementations in [6–8], please refer to [8, pages 120–125] for details. So, the algorithm in [6–8] to generate the steplength satisfying the approximate Wolfe conditions (8) is not applicable. Here, we determine the steplength 𝛼_𝑘 of the MDYHS+1 method following the inexact line search strategies of [15, Algorithm 2.6]. Detailed steps are described in Algorithm 6. The initial value of 𝛼_𝑘 is taken to be 𝜌_𝑘.

Remark 5. The choice of 𝜌_𝑘 comes from [13]. Since it is important to use current information about the algorithm and the problem to make an initial guess of 𝛼^∗_𝑘, the author of [13] uses the relation 𝑔(𝑥_𝑘+𝛼^∗_𝑘𝑑_𝑘)^𝑇𝑑_𝑘=0 and 𝑔𝑥_𝑘+𝛼^∗_𝑘𝑑_𝑘−𝑔_𝑘^𝑇𝑑_𝑘𝛼^∗_𝑘≈𝑔𝑥_𝑘+𝛼_𝑘−1𝑑_𝑘−𝑔_𝑘^𝑇𝑑_𝑘𝛼_𝑘−1 (17) to give an approximation to the optimal steplength through 𝛼^∗_𝑘≈−𝑔^𝑇_𝑘𝑑_𝑘𝜇_𝑘𝑔_𝑘²𝑔_𝑘²𝑑_𝑘². (18)

Now, we describe the line search algorithm in [15], which is very close to one suggested in [16].

Algorithm 6. Step 0. Set 𝑢=0 and 𝑣=+∞. Choose 𝛼>0. Set 𝑗∶=0.

Step 1. If 𝛼 does not satisfy 𝑔𝑥_𝑘+𝛼𝑑_𝑘^𝑇𝑑_𝑘≤(2𝛿−1)𝑔^𝑇_𝑘𝑑_𝑘, (19) then set 𝑗∶=𝑗+1, and go to Step 2. If 𝛼 does not satisfy 𝜎𝑔^𝑇_𝑘𝑑_𝑘≤𝑔𝑥_𝑘+𝛼𝑑_𝑘^𝑇𝑑_𝑘, (20) then set 𝑗∶=𝑗+1, and go to Step 3. Otherwise, set 𝛼_𝑘∶=𝛼, and return.

Step 2. Set 𝑣=𝛼,𝛼=(𝑢+𝑣)/2. Then go to Step 1.

Step 3. Set 𝑢=𝛼,𝛼=2𝑢. Then go to Step 1.

Next, we analyze the convergence properties of the MDYHS+ method.

Lemma 7. Consider the previous MDYHS+ method. If 𝑔_𝑘≠0 for all 𝑘≥0, then the steplength 𝛼_𝑘 is well-defined, namely, 𝛼_𝑘 can be found to satisfy (16) after a finite number of trials. The search direction 𝑑_𝑘 satisfies the sufficient descent condition 𝑔^𝑇_𝑘𝑑_𝑘≤−𝑔_𝑘²<0. (21) Furthermore, 𝑔^𝑇_𝑘+1𝑑_𝑘<0.

Proof. We prove the desired results by induction. When 𝑘=0, by using 𝑑₀=−𝑔₀, we have 𝑔^𝑇₀𝑑₀=−𝑔₀²<0. (22) We now show that the steplength 𝛼₀ will be determined within a finite number of trials. Otherwise, for any 𝑗=0,1,2,…, the following inequality holds 𝑔𝑥₀+𝜌₀𝑡^𝑗𝑑₀^𝑇𝑑₀+12max−𝜇₀,0𝜌₀𝑡^𝑗𝑑₀²>𝜎𝑔^𝑇₀𝑑₀, (23) where 0<𝜎<1. Since 𝑔 is continuous, taking the limits with respect to 𝑗 on the both sides of (23) yields 𝑔^𝑇₀𝑑₀≥𝜎𝑔^𝑇₀𝑑₀, (24) this is a contradiction. Then, 𝛼₀ is well-defined and 𝑔^𝑇₁𝑑₀=𝑔𝑥₀+𝛼₀𝑑₀^𝑇𝑑₀≤𝜎𝑔^𝑇₀𝑑₀<0. (25) Assume that (21) holds for 𝑘−1 and 𝑔^𝑇_𝑘𝑑_𝑘−1<0. A similar discussion to the 𝑘=0 case yields 𝛼_𝑘 is well-defined. Multiplying 𝑔^𝑇_𝑘 by 𝑑_𝑘=−𝑔_𝑘+𝛽^DYHS+_𝑘𝑑_𝑘−1, (26) we have that 𝑔^𝑇_𝑘𝑑_𝑘=−𝑔_𝑘²+𝛽^DYHS+_𝑘𝑔^𝑇_𝑘𝑑_𝑘−1, (27) it follows from 𝑔^𝑇_𝑘𝑑_𝑘−1<0 and 𝛽^DYHS+_𝑘≥0 that 𝑔^𝑇_𝑘𝑑_𝑘≤−𝑔_𝑘²<0. (28) Then, 𝑔^𝑇_𝑘+1𝑑_𝑘≤𝑔^𝑇_𝑘+1𝑑_𝑘+12max−𝜇_𝑘,0𝛼_𝑘𝑑_𝑘²≤𝜎𝑔^𝑇_𝑘𝑑_𝑘<0. (29)

Theorem 8. Consider the previos MDYHS+ method, and assume that 𝑔 is 𝜇_𝑘-monotone in the interval {𝑥_𝑘+𝛼𝑑_𝑘∶0≤𝛼≤𝜌_𝑘}, where 𝜇_𝑘 is the very 𝜇_𝑘 defined in (12). If Assumption 1 holds, then either liminf_𝑘→+∞𝑔_𝑘=0, (30) or _𝑘≥0𝑔^𝑇_𝑘𝑑_𝑘²𝑑_𝑘²<+∞. (31)

Proof. Since 𝑔 is 𝜇_𝑘-monotone, by using Lemma 3, we obtain 𝑓𝑥_𝑘+𝛼_𝑘𝑑_𝑘≤𝑓𝑥_𝑘+𝛼_𝑘𝑔𝑥_𝑘+𝛼_𝑘𝑑_𝑘^𝑇𝑑_𝑘−12𝜇_𝑘𝛼²_𝑘𝑑_𝑘². (32) Furthermore, 𝑓𝑥_𝑘+𝛼_𝑘𝑑_𝑘≤𝑓𝑥_𝑘+𝛼_𝑘𝑔𝑥_𝑘+𝛼_𝑘𝑑_𝑘^𝑇𝑑_𝑘+12max−𝜇_𝑘,0𝛼²_𝑘𝑑_𝑘². (33) Combining (33) with (16) yields 𝑓𝑥_𝑘+𝛼_𝑘𝑑_𝑘≤𝑓𝑥_𝑘+𝜎𝛼_𝑘𝑔^𝑇_𝑘𝑑_𝑘, (34) which, together with (21), implies ^𝑘_𝑖=0𝛼_𝑖𝑔_𝑖²≤𝜎⁻¹𝑓𝑥₀−𝑓𝑥_𝑘+1. (35)
Next, we follow [13] to consider two possible cases.
Case 1 (∑^+∞_𝑘=0𝛼_𝑘=+∞). Consider Assumption 1, we have min_{0≤𝑖≤𝑘}𝑔_𝑖²≤𝜎⁻¹𝑓𝑥₀−𝑓𝑥_𝑘+1^𝑘_𝑖=0𝛼_𝑖⁻¹⟶0,as𝑘⟶+∞. (36) Thus, (30) holds.
Case 2 (∑^+∞_𝑘=0𝛼_𝑘<+∞). From this case, we have 𝛼_𝑘→0. Since 𝛼_𝑘 is the maximal number in {𝜌_𝑘𝑡^𝑗∶𝑗=0,1,2,…} such that (16) holds, then there must exist a natural number 𝑁>0, when 𝑘≥𝑁, there exists 𝑗_𝑘>1 such that 𝛼_𝑘=𝜌_𝑘𝑡^𝑗𝑘 and 𝑡⁻¹𝛼_𝑘 violates (16), namely, 𝑔𝑥_𝑘+𝑡⁻¹𝛼_𝑘𝑑_𝑘^𝑇𝑑_𝑘+12max−𝜇_𝑘,0𝑡⁻¹𝛼_𝑘𝑑_𝑘²>𝜎𝑔^𝑇_𝑘𝑑_𝑘, (37) then 𝑔𝑥_𝑘+𝑡⁻¹𝛼_𝑘𝑑_𝑘−𝑔_𝑘^𝑇𝑑_𝑘𝑡⁻¹𝑑_𝑘²+12max−𝜇_𝑘,0𝑡⁻¹𝛼_𝑘𝑑_𝑘²𝑡⁻¹𝑑_𝑘²≥(𝜎−1)𝑡𝑔^𝑇_𝑘𝑑_𝑘𝑑_𝑘². (38) Combining it with the Lipschitz continuity of 𝑔 and the fact that |𝜇_𝑘|≤𝐿 yields 𝛼_𝑘≥−2(1−𝜎)𝑡𝑔^𝑇_𝑘𝑑_𝑘3𝐿𝑑_𝑘², (39) which, together with (34), implies 𝑓𝑥_𝑘−𝑓𝑥_𝑘+1𝑔≥𝛾^𝑇𝑘𝑑_𝑘²𝑑_𝑘², (40) with 𝛾=2𝜎(1−𝜎)𝑡/(3𝐿)>0. Using (40) recursively, we obtain 𝛾^𝑘_𝑖=𝑁𝑔^𝑇_𝑘𝑑_𝑘²𝑑_𝑘²≤𝑓𝑥_𝑁−𝑓𝑥_𝑘+1. (41) Combining it with Assumption 1 yields _𝑘≥𝑁𝑔^𝑇_𝑘𝑑_𝑘²𝑑_𝑘²<+∞, (42) namely, _𝑘≥0𝑔^𝑇_𝑘𝑑_𝑘²𝑑_𝑘²<+∞. (43)

Theorem 9. The MDYHS+ method is convergent in the sense that liminf_𝑘→∞‖𝑔_𝑘‖=0.

Proof. The proof of Theorem 9 is much like the convergence analysis of the DY method [1, Theorem 3.3], so the corresponding details are omitted here.

4. Numerical Experiments

In this section, we did some numerical experiments to test the performances of the MDYHS+ method and the MDYHS + 1 method. One purpose of this section is to compare them with the DYHS method and the DYHS+ method. The other purpose is to confirm their broader application scope by solving boundary value problems. So, two classes of test problems were selected here. One class was drawn from the CUTEr library [17, 18], and the other class came from [19]. More information was described in the following subsections.

For the MDYHS+ method, we set 𝜎=10⁻⁴ and 𝑡=0.5. For the MDYHS + 1 method, we followed [8] to choose 𝜎=0.9 and 𝛿=0.1. And for the hybrid conjugate gradient methods DYHS and DYHS+, the values of 𝛿 and 𝜎 in (5) and (6) were taken to be 0.01 and 0.1, respectively. The initial value of 𝛼_𝑘 was taken to be 1/‖𝑔₀‖ for the first iteration and 𝛼_𝑘−1𝑔^𝑇_𝑘−1𝑑_𝑘−1/(𝑔^𝑇_𝑘𝑑_𝑘) for 𝑘≥1 (see [10]). For all the methods, the largest trial times of choosing steplength at each iteration was taken to be 𝑗_max=30, and the stopping criterion used was ‖𝑔_𝑘‖_∞≤𝜖. In order to understand the numerical performance of each method deeply, we did numerical experiments with 𝜖=10^−𝑖,𝑖=3,6,9,12. Our computations were carried out using Matlab R2012a on a desktop computer with an Intel(R) Xeon(R) 2.40 GHZ CPU and 6.00 GB of RAM. The operating system is Linux: Ubuntu 8.04.

4.1. Tested by Some Problems in the CUTEr Library

In this subsection, we implemented four different hybrid conjugate gradient methods and compared their numerical performances. Because the DYHS method and the DYHS+ method need the information of the original function of (1), we selected a collection of test problems from the CUTEr library and listed them in Table 1. The first column “Prob.” denotes the problem number, and the columns “Name” and “𝑛” denote the name and the dimension of the problem, respectively. Since we were interested in large-scale problems, we only considered problems with size at least 100. The largest dimension was set to 10,000. Moreover, we accessed CUTEr functions from within Matlab R2012a by using Matlab interface.

Table 1 

List of test problems from the CUTEr library.

Our numerical results were reported in Tables 2, 3, 4, 5, and 6 in the form of 𝑘/𝐼𝑛𝑛𝑒𝑟𝑘/𝐶𝑃𝑈, where 𝑘, 𝐼𝑛𝑛𝑒𝑟𝑘, and 𝐶𝑃𝑈 stand for the number of iterations, the total trial times of the line search and the CPU time elapsed, respectively. For the DYHS+ and the DYHS, we let 𝑁_𝑓 and 𝑁_𝑔 be the number of function evaluations and the number of gradient evaluations, respectively, and set 𝐼𝑛𝑛𝑒𝑟𝑘=𝑁_𝑓/3+𝑁_𝑔 by automatic differentiation (see [10, 20] for details). Moreover, “—” means the method’s failure to achieve a prescribed accuracy when the number of iterations exceeded 50,000, and the test problems are represented in the form of #Pro.(𝑛).

Table 2 

Numerical results for test problems from the CUTEr library.

Table 3 

Numerical results for test problems from the CUTEr library.

Table 4 

Numerical results for test problems from the CUTEr library.

Table 5 

Numerical results for test problems from the CUTEr library.

Table 6 

Numerical results for test problems from the CUTEr library.

The performances of the four methods, relative to CPU time, were evaluated using the profiles of Dolan and Morè [21]. That is, for the four methods, we plotted the fraction 𝑃 of problems for which each of the methods was within a factor 𝜏 of the best time. Figures 1, 2, and 3 showed the performance profiles referring to CPU time, the number of iterations and the total trial times of the line search, respectively. These figures revealed that the MDYHS+ method and the MDYHS + 1 method performed better than the DYHS method and the DYHS+ method. The performance profiles also showed that the MDYHS+ method and the MDYHS + 1 method were comparable and solved almost all of the test problems up to 𝜖=10⁻⁹. Yet, the latter has no convergence.

(a)

(b)

Figure 1 

Performance profiles based on CPU time.

(a)

(b)

Figure 2 

Performance profiles based on the number of iterations.

(a)

(b)

Figure 3 

Performance profiles based on the number of inner iterations.

4.2. Tested by Some Boundary Value Problems

In this section, we implemented the MDYHS+ method and the MDYHS + 1 method to solve some boundary value problems. See [22, Chapter 1] for the background of the boundary value problems.

In order to confirm the efficiency of the MDYHS+ method and the MDYHS + 1 method to solve this class of problems, We drew a set of 11 boundary value problems from [19] and listed them in Table 7, where the test problems were expressed by #Pro.(𝑛) (#Pro. denotes the problem number in [19] and 𝑛 denotes the dimension), and the test results were listed in the form of 𝑘/𝐼𝑛𝑛𝑒𝑟𝑘/𝐶𝑃𝑈.

Table 7 

Numerical results for some boundary value problems.

From Table 7, we can see that both of the MDYHS+ method and the MDYHS + 1 method are efficient in solving boundary value problems. The MDYHS + 1 method seems a little better but has no convergence.

5. Conclusions

This paper has studied two modified versions of a Dai-Yuan hybrid conjugate gradient method with two different line searches only using gradient information and has proven that with the line search (9), it is convergent in the sense of liminf_𝑘→∞‖𝑔_𝑘‖=0. Then, we investigated the numerical behaviors of the two modified versions over two classes of standard test problems. From the numerical results, we can conclude that the two modified hybrid conjugate gradient methods are more efficient (especially in high precision) in solving large-scale nonlinear unconstrained minimization problems and have broader application scope. For example, they can be used to solve some boundary value problems, where functions are not explicit.

Acknowledgments

The authors are very grateful to the associate editor and the referees for their valuable suggestions. Meanwhile, the first author is also very grateful to Yunda Dong for suggesting her to write this paper and to add Section 4.2 to the revised version. This work was supported by National Science Foundation of China, no. 60974082.