Abstract

A modified PRP nonlinear conjugate gradient method to solve unconstrained optimization problems is proposed. The important property of the proposed method is that the sufficient descent property is guaranteed independent of any line search. By the use of the Wolfe line search, the global convergence of the proposed method is established for nonconvex minimization. Numerical results show that the proposed method is effective and promising by comparing with the VPRP, CG-DESCENT, and DL+ methods.

1. Introduction

The nonlinear conjugate gradient method is one of the most efficient methods in solving unconstrained optimization problems. It comprises a class of unconstrained optimization algorithms which is characterized by low memory requirements and simplicity.

Consider the unconstrained optimization problem minπ‘₯βˆˆπ‘…π‘›π‘“(π‘₯),(1.1) where π‘“βˆΆπ‘…π‘›β†’π‘… is continuously differentiable, and its gradient 𝑔 is available.

The iterates of the conjugate gradient method for solving (1.1) are given by π‘₯π‘˜+1=π‘₯π‘˜+π›Όπ‘˜π‘‘π‘˜,(1.2) where stepsize π›Όπ‘˜ is positive and computed by certain line search, and the search direction π‘‘π‘˜ is defined by π‘‘π‘˜=ξƒ―βˆ’π‘”π‘˜,forπ‘˜=1,βˆ’π‘”π‘˜+π›½π‘˜π‘‘π‘˜βˆ’1,forπ‘˜β‰₯2,(1.3) where π‘”π‘˜=βˆ‡π‘“(π‘₯π‘˜), and π›½π‘˜ is a scalar. Some well-known conjugate gradient methods include Polak-RibiΓ¨re-Polyak (PRP) method [1, 2], Hestenes-Stiefel (HS) method [3], Hager-Zhang (HZ) method [4], and Dai-Liao (DL) method [5]. The parameters π›½π‘˜ of these methods are specified as follows: 𝛽PRPπ‘˜=π‘”π‘‡π‘˜ξ€·π‘”π‘˜βˆ’π‘”π‘˜βˆ’1ξ€Έβ€–β€–π‘”π‘˜βˆ’1β€–β€–2,𝛽HSπ‘˜=π‘”π‘‡π‘˜ξ€·π‘”π‘˜βˆ’π‘”π‘˜βˆ’1ξ€Έπ‘‘π‘‡π‘˜βˆ’1ξ€·π‘”π‘˜βˆ’π‘”π‘˜βˆ’1ξ€Έ,𝛽HZπ‘˜=ξƒ©π‘¦π‘˜βˆ’1βˆ’2π‘‘π‘˜βˆ’1β€–β€–π‘¦π‘˜βˆ’1β€–β€–2π‘‘π‘‡π‘˜βˆ’1π‘¦π‘˜βˆ’1ξƒͺπ‘‡π‘”π‘˜π‘‘π‘‡π‘˜βˆ’1π‘¦π‘˜βˆ’1,𝛽DLπ‘˜=π‘”π‘‡π‘˜π‘¦π‘˜βˆ’1π‘‘π‘‡π‘˜βˆ’1π‘¦π‘˜βˆ’1π‘”βˆ’π‘‘π‘‡π‘˜π‘ π‘˜βˆ’1π‘‘π‘‡π‘˜βˆ’1π‘¦π‘˜βˆ’1,(𝑑β‰₯0),(1.4) where ||β‹…|| is the Euclidean norm and π‘¦π‘˜βˆ’1=π‘”π‘˜βˆ’π‘”π‘˜βˆ’1. We know that if 𝑓 is a strictly convex quadratic function, the above methods are equivalent in the case that an exact line search is used. If 𝑓 is nonconvex, their behaviors may be further different.

In the past few years, the PRP method has been regarded as the most efficient conjugate gradient method in practical computation. One remarkable property of the PRP method is that it essentially performs a restart if a bad direction occurs (see [6]). Powell [7] constructed an example which showed that the PRP method can cycle infinitely without approaching any stationary point even if an exact line search is used. This counterexample also indicates that the PRP method has a drawback that it may not globally be convergent when the objective function is nonconvex. Powell [8] suggested that the parameter π›½π‘˜ is negative in the PRP method and defined π›½π‘˜ as π›½π‘˜ξ€½=max0,𝛽PRPπ‘˜ξ€Ύ.(1.5) Gilbert and Nocedal [9] considered Powell’s suggestion and proved the global convergence of the modified PRP method for nonconvex functions under the appropriate line search. In addition, there are many researches on convergence properties of the PRP method (see [10–12]).

In recent years, much effort has been investigated to create new methods, which not only possess global convergence properties for general functions but also are superior to original methods from the computation point of view. For example, Yu et al. [13] proposed a new nonlinear conjugate gradient method in which the parameter π›½π‘˜ is defined on the basic of 𝛽PRPπ‘˜ such as 𝛽VPRPπ‘˜=⎧βŽͺ⎨βŽͺβŽ©β€–β€–π‘”π‘˜β€–β€–2βˆ’||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||𝜈||π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1||+β€–β€–π‘”π‘˜βˆ’1β€–β€–2‖‖𝑔ifπ‘˜β€–β€–2>||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||,0,otherwise,(1.6) where 𝜈>1 (in this paper, we call this method as VPRP method). And they proved the global convergence of the VPRP method with the Wolfe line search. Hager and Zhang [4] discussed the global convergence of the HZ method for strong convex functions under the Wolfe line search and Goldstein line search. In order to prove the global convergence for general functions, Hager and Zhang modified the parameter 𝛽HZπ‘˜ as 𝛽MHZπ‘˜ξ€½π›½=maxHZπ‘˜,πœ‚π‘˜ξ€Ύ,(1.7) where πœ‚π‘˜=βˆ’1β€–β€–π‘‘π‘˜β€–β€–ξ€½β€–β€–π‘”minπœ‚,π‘˜β€–β€–ξ€Ύ,πœ‚>0.(1.8) The corresponding method of (1.7) is the famous CG-DESCENT method.

Dai and Liao [5] proposed a new conjugate condition, that is, π‘‘π‘‡π‘˜π‘¦π‘˜βˆ’1=βˆ’π‘‘π‘”π‘‡π‘˜π‘ π‘˜βˆ’1,(𝑑β‰₯0).(1.9) Under the new conjugate condition, they proved global convergence of the DL conjugate gradient method for uniformly convex functions. According to Powell’s suggestion, Dai and Liao gave a modified parameter π›½π‘˜ξƒ―π‘”=maxπ‘‡π‘˜π‘¦π‘˜βˆ’1π‘‘π‘‡π‘˜βˆ’1π‘¦π‘˜βˆ’1𝑔,0βˆ’π‘‘π‘‡π‘˜π‘ π‘˜βˆ’1π‘‘π‘‡π‘˜βˆ’1π‘¦π‘˜βˆ’1,(𝑑β‰₯0).(1.10) The corresponding method of (1.10) is the famous DL+ method. Under the strong Wolfe line search, they researched the global convergence of the DL+ method for general functions. Zhang et al. [14] proposed a modified DL conjugate gradient method and proved its global convergence. Moreover, some researchers have been studying a new type of method called the spectral conjugate gradient method (see [15–17]).

This paper is organized as follows: in the next section, we propose a modified PRP method and prove its sufficient descent property. In Section 3, the global convergence of the method with the Wolfe line search is given. In Section 4, numerical results are reported. We have a conclusion in the last section.

2. Modified PRP Method

In this section, we propose a modified PRP conjugate gradient method in which the parameter π›½π‘˜ is defined on the basic of 𝛽PRPπ‘˜ as follows: 𝛽MPRPπ‘˜=⎧βŽͺ⎨βŽͺβŽ©β€–β€–π‘”π‘˜β€–β€–2βˆ’||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||ξ€½max0,π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1ξ€Ύ+β€–β€–π‘”π‘˜βˆ’1β€–β€–2‖‖𝑔ifπ‘˜β€–β€–2β‰₯||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||‖‖𝑔β‰₯π‘šπ‘˜β€–β€–2,0,otherwise,(2.1) in which π‘šβˆˆ(0,1). We introduce the modified PRP method as follows.

2.1. Modified PRP (MPRP) Method

Step 1. Set π‘₯1βˆˆπ‘…π‘›, πœ€β‰₯0, and 𝑑1=βˆ’π‘”1, if ‖𝑔1β€–β‰€πœ€, then stop.

Step 2. Compute π›Όπ‘˜ by some inexact line search.

Step 3. Let π‘₯π‘˜+1=π‘₯π‘˜+π›Όπ‘˜π‘‘π‘˜, π‘”π‘˜+1=𝑔(π‘₯π‘˜+1), if ||π‘”π‘˜+1||β‰€πœ€, then stop.

Step 4. Compute π›½π‘˜+1 by (2.1), and generate π‘‘π‘˜+1 by (1.3).

Step 5. Set π‘˜=π‘˜+1, and go to Step 2.

In the convergence analyses and implementations of conjugate gradient methods, one often requires the inexact line search to satisfy the Wolfe line search or the strong Wolfe line search. The Wolfe line search is to find π›Όπ‘˜ such that 𝑓π‘₯π‘˜+π›Όπ‘˜π‘‘π‘˜ξ€Έξ€·π‘₯β‰€π‘“π‘˜ξ€Έ+π›Ώπ›Όπ‘˜π‘”π‘‡π‘˜π‘‘π‘˜π‘”ξ€·π‘₯,(2.2)π‘˜+π›Όπ‘˜π‘‘π‘˜ξ€Έπ‘‡π‘‘π‘˜β‰₯πœŽπ‘”π‘‡π‘˜π‘‘π‘˜,(2.3) where 0<𝛿<𝜎<1. The strong Wolfe line search consists of (2.2) and the following strengthened version of (2.3): |||𝑔π‘₯π‘˜+π›Όπ‘˜π‘‘π‘˜ξ€Έπ‘‡π‘‘π‘˜|||β‰€βˆ’πœŽπ‘”π‘‡π‘˜π‘‘π‘˜.(2.4)

Moreover, in most references, we can see that the sufficient descent condition π‘”π‘‡π‘˜π‘‘π‘˜β€–β€–π‘”β‰€βˆ’π‘π‘˜β€–β€–2,𝑐>0(2.5) is always given which plays a vital role in guaranteeing the global convergence properties of conjugate gradient methods. But, in this paper, π‘‘π‘˜ can satisfy (2.5) without any line search.

Theorem 2.1. Consider any method (1.2)-(1.3), where π›½π‘˜=𝛽MPRPπ‘˜. If π‘”π‘˜β‰ 0 for all π‘˜β‰₯1, then π‘”π‘‡π‘˜π‘‘π‘˜β€–β€–g<βˆ’π‘˜β€–β€–2,βˆ€π‘˜β‰₯1.(2.6)

Proof. Multiplying (1.3) by π‘”π‘‡π‘˜, we get π‘”π‘‡π‘˜π‘‘π‘˜β€–β€–π‘”=βˆ’π‘˜β€–β€–2+𝛽MPRPπ‘˜π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1.(2.7) If 𝛽MPRPπ‘˜=0, from (2.7), we know that the conclusion (2.6) holds. If 𝛽MPRPπ‘˜β‰ 0, the proof is divided into two cases in the following.
Firstly, if π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1≀0, then from (2.1) and (2.7), one has π‘”π‘‡π‘˜π‘‘π‘˜β€–β€–π‘”=βˆ’π‘˜β€–β€–2+β€–β€–π‘”π‘˜β€–β€–2βˆ’||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||ξ€½max0,π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1ξ€Ύ+β€–β€–π‘”π‘˜βˆ’1β€–β€–2β‹…π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1‖‖𝑔=βˆ’π‘˜β€–β€–2+β€–β€–π‘”π‘˜β€–β€–2βˆ’||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||β€–β€–π‘”π‘˜βˆ’1β€–β€–2β‹…π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1=ξ‚€βˆ’β€–β€–π‘”π‘˜β€–β€–2⋅||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||/β€–β€–π‘”π‘˜β€–β€–2ξ‚β‹…π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1βˆ’π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1+β€–β€–π‘”π‘˜βˆ’1β€–β€–2β€–β€–π‘”π‘˜βˆ’1β€–β€–2=ξ‚€βˆ’β€–β€–π‘”π‘˜β€–β€–2ξ‚β‹…β€–β€–π‘”π‘˜βˆ’1β€–β€–2βˆ’π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1ξ‚€||𝑔1βˆ’π‘‡π‘˜π‘”π‘˜βˆ’1||/β€–β€–π‘”π‘˜β€–β€–2ξ‚β€–β€–π‘”π‘˜βˆ’1β€–β€–2β‰€ξ‚€βˆ’β€–β€–π‘”π‘˜β€–β€–2ξ‚β‹…β€–β€–π‘”π‘˜βˆ’1β€–β€–2β€–β€–π‘”π‘˜βˆ’1β€–β€–2‖‖𝑔=βˆ’π‘˜β€–β€–2<0.(2.8)
Secondly, if π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1>0, then from (2.7), we also have π‘”π‘‡π‘˜π‘‘π‘˜β€–β€–π‘”<βˆ’π‘˜β€–β€–2+β€–β€–π‘”π‘˜β€–β€–2βˆ’||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1β‹…π‘”π‘‡π‘˜π‘‘π‘˜βˆ’1||𝑔=βˆ’π‘‡π‘˜π‘”π‘˜βˆ’1||β€–β€–π‘”β‰€βˆ’π‘šπ‘˜β€–β€–2.(2.9) From the above, the conclusion (2.6) holds under any line search.

3. Global Convergences of the Modified PRP Method

In order to prove the global convergence of the modified PRP method, we assume that the objective function 𝑓(π‘₯) satisfies the following assumption.

Assumption H
(i) The level set Ξ©={π‘₯βˆˆπ‘…π‘›βˆ£π‘“(π‘₯)≀𝑓(π‘₯1)} is bounded, that is, there exists a positive constant πœ‰>0 such that for all π‘₯∈Ω, ||π‘₯||β‰€πœ‰.
(ii) In a neighborhood 𝑉 of Ξ©, 𝑓 is continuously differentiable and its gradient 𝑔 is Lipchitz continuous, namely, there exists a constant 𝐿>0 such that ‖𝑔(π‘₯)βˆ’π‘”(𝑦)‖≀𝐿‖π‘₯βˆ’π‘¦β€–,βˆ€π‘₯,π‘¦βˆˆπ‘‰.(3.1) Under these assumptions on 𝑓, there exists a constant 𝛾>0 such that ‖𝑔(π‘₯)β€–β‰€π›Ύβˆ€π‘₯∈Ω.(3.2)
The conclusion of the following lemma, often called the Zoutendijk condition, is used to prove the global convergence properties of nonlinear conjugate gradient methods. It was originally given by Zoutendijk [18].

Lemma 3.1. Suppose that, Assumption H holds. Consider any iteration of (1.2)-(1.3), where π‘‘π‘˜ satisfies π‘”π‘‡π‘˜π‘‘π‘˜<0 for π‘˜βˆˆπ‘+ and π›Όπ‘˜ satisfies the Wolfe line search, then ξ“π‘˜β‰₯1ξ€·π‘”π‘‡π‘˜π‘‘π‘˜ξ€Έ2β€–β€–π‘‘π‘˜β€–β€–2<+∞.(3.3)

Lemma 3.2. Suppose that Assumption H holds. Consider the method (1.2)-(1.3), where 𝛽k=𝛽MPRPπ‘˜, and 𝛼k satisfies the Wolfe line search and (2.6). If there exists a constant π‘Ÿ>0, such that β€–β€–π‘”π‘˜β€–β€–β‰₯π‘Ÿ,βˆ€π‘˜β‰₯1,(3.4) then one has ξ“π‘˜β‰₯2β€–β€–π‘’π‘˜βˆ’π‘’π‘˜βˆ’1β€–β€–2<+∞,(3.5) where π‘’π‘˜=π‘‘π‘˜/||π‘‘π‘˜||.

Proof. From (2.1) and (3.4), we get π‘”π‘‡π‘˜π‘”π‘˜βˆ’1β‰ 0.(3.6) By (2.6) and (3.6), we know that π‘‘π‘˜β‰ 0 for each π‘˜.
Define the quantities π‘Ÿπ‘˜=βˆ’π‘”π‘˜β€–β€–π‘‘π‘˜β€–β€–,π›Ώπ‘˜=𝛽MPRPπ‘˜β€–β€–π‘‘π‘˜βˆ’1β€–β€–β€–β€–π‘‘π‘˜β€–β€–.(3.7) By (1.3), one has π‘’π‘˜=π‘‘π‘˜β€–β€–π‘‘π‘˜β€–β€–=βˆ’π‘”π‘˜+𝛽MPRPπ‘˜π‘‘π‘˜βˆ’1β€–β€–π‘‘π‘˜β€–β€–=π‘Ÿπ‘˜+π›Ώπ‘˜π‘’π‘˜βˆ’1.(3.8) Since π‘’π‘˜ is unit vector, we get β€–β€–π‘Ÿπ‘˜β€–β€–=β€–β€–π‘’π‘˜βˆ’π›Ώπ‘˜π‘’π‘˜βˆ’1β€–β€–=β€–β€–π›Ώπ‘˜π‘’π‘˜βˆ’π‘’π‘˜βˆ’1β€–β€–.(3.9) From π›Ώπ‘˜β‰₯0 and the above equation, one has β€–β€–π‘’π‘˜βˆ’π‘’π‘˜βˆ’1‖‖≀1+π›Ώπ‘˜ξ€Έβ€–β€–π‘’π‘˜βˆ’π‘’π‘˜βˆ’1β€–β€–=β€–β€–ξ€·1+π›Ώπ‘˜ξ€Έπ‘’π‘˜βˆ’ξ€·1+π›Ώπ‘˜ξ€Έπ‘’π‘˜βˆ’1β€–β€–β‰€β€–β€–π‘’π‘˜βˆ’π›Ώπ‘˜π‘’π‘˜βˆ’1β€–β€–+β€–β€–π›Ώπ‘˜π‘’π‘˜βˆ’π‘’π‘˜βˆ’1β€–β€–β€–β€–π‘Ÿ=2π‘˜β€–β€–.(3.10) By (2.1), (3.4), and (3.6), one has ||𝑔1β‰₯π‘‡π‘˜π‘”π‘˜βˆ’1||2β€–β€–π‘”π‘˜β€–β€–2>π‘š.(3.11) From (3.3), (2.6), (3.4), and (3.11), one has π‘š2ξ“π‘˜β‰₯1,π‘‘π‘˜β‰ 0β€–β€–π‘Ÿπ‘˜β€–β€–2β‰€ξ“π‘˜β‰₯1,π‘‘π‘˜β‰ 0ξƒ©β€–β€–π‘Ÿπ‘˜β€–β€–2β‹…||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||2β€–β€–π‘”π‘˜β€–β€–2ξƒͺ=ξ“π‘˜β‰₯1,π‘‘π‘˜β‰ 0||π‘”π‘‡π‘˜π‘”π‘˜βˆ’1||2β€–β€–π‘‘π‘˜β€–β€–2β‰€ξ“π‘˜β‰₯1,π‘‘π‘˜β‰ 0ξ€·π‘”π‘‡π‘˜π‘‘π‘˜ξ€Έ2β€–β€–π‘‘π‘˜β€–β€–2<+∞,(3.12) so ξ“π‘˜β‰₯1,π‘‘π‘˜β‰ 0β€–β€–π‘Ÿπ‘˜β€–β€–2<+∞.(3.13) By (3.10) and the above inequality, one has ξ“π‘˜β‰₯2β€–β€–π‘’π‘˜βˆ’π‘’π‘˜βˆ’1β€–β€–2<+∞.(3.14)

Lemma 3.3. Suppose that Assumption H holds. If (3.4) holds, then 𝛽MPRPπ‘˜ has property (*), that is, (1)there exists a constant 𝑏>1, such that |𝛽MPRPπ‘˜|≀𝑏,(2)there exists a constant πœ†>0, such that ||π‘₯π‘˜βˆ’π‘₯π‘˜βˆ’1||β‰€πœ†β‡’|𝛽MPRPπ‘˜|≀1/2𝑏.

Proof. From Assumption (ii), we know that (3.2) holds. By (2.1), (3.2), and (3.4), one has ||𝛽MPRPπ‘˜||β‰€ξ€·β€–β€–π‘”π‘˜β€–β€–+β€–β€–π‘”π‘˜βˆ’1β€–β€–ξ€Έβ‹…β€–β€–π‘”π‘˜β€–β€–β€–β€–π‘”π‘˜βˆ’1β€–β€–2≀2𝛾2π‘Ÿ2=𝑏.(3.15) Define πœ†=π‘Ÿ2/2𝐿𝛾𝑏. If ||π‘₯π‘˜βˆ’π‘₯π‘˜βˆ’1||β‰€πœ†, then from (2.1), (3.1), (3.2), and (3.4), one has ||𝛽MPRPπ‘˜||β‰€β€–β€–π‘”π‘˜β€–β€–2βˆ’π‘”π‘‡π‘˜π‘”π‘˜βˆ’1β€–β€–π‘”π‘˜βˆ’1β€–β€–2=π‘”π‘‡π‘˜ξ€·π‘”π‘˜βˆ’π‘”π‘˜βˆ’1ξ€Έβ€–β€–π‘”π‘˜βˆ’1β€–β€–2β‰€β€–β€–π‘”π‘˜β€–β€–β‹…β€–β€–π‘”π‘˜βˆ’π‘”π‘˜βˆ’1β€–β€–β€–β€–π‘”π‘˜βˆ’1β€–β€–2β‰€π›ΎπΏπœ†π‘Ÿ2=1.2𝑏(3.16)

Lemma 3.4 (see [19]). Suppose that Assumption H holds. Let {π‘₯π‘˜} and {π‘‘π‘˜} be generated by (1.2)-(1.3), in which π›Όπ‘˜ satisfies the Wolfe line search and (2.6). If π›½π‘˜β‰₯0 has the property (*) and (3.4) holds, then there exits πœ†>0, for any Ξ”βˆˆπ‘+ and π‘˜0βˆˆπ‘+, for all π‘˜β‰₯π‘˜0, such that ||β„œπœ†π‘˜,Ξ”||>Ξ”2,(3.17) where β„œπœ†π‘˜,Ξ”β‰œ{π‘–βˆˆπ‘+βˆΆπ‘˜β‰€π‘–β‰€π‘˜+Ξ”βˆ’1,||π‘₯π‘–βˆ’π‘₯π‘–βˆ’1||β‰₯πœ†}, |β„œπœ†π‘˜,Ξ”| denotes the number of the β„œπœ†π‘˜,Ξ”.

Theorem 3.5. Suppose that Assumption H holds. Let {π‘₯π‘˜} and {π‘‘π‘˜} be generated by (1.2)-(1.3), in which π›Όπ‘˜ satisfies the Wolfe line search and (2.6), π›½π‘˜=𝛽MPRPπ‘˜, then one has liminfπ‘˜β†’+βˆžβ€–β€–π‘”π‘˜β€–β€–=0.(3.18)

Proof. To obtain this result, we proceed by contradiction. Suppose that (3.18) does not hold, which means that there exists π‘Ÿ>0 such that β€–β€–π‘”π‘˜β€–β€–β‰₯π‘Ÿ,forπ‘˜β‰₯1,(3.19) so, we know that Lemmas 3.2 and 3.4 hold.
We also define π‘’π‘˜=π‘‘π‘˜/||π‘‘π‘˜||, then for all 𝑙,π‘˜βˆˆπ‘+(𝑙β‰₯π‘˜), one has π‘₯π‘™βˆ’π‘₯π‘˜βˆ’1=𝑙𝑖=π‘˜β€–β€–π‘₯π‘–βˆ’π‘₯π‘–βˆ’1β€–β€–β‹…π‘’π‘–βˆ’1=𝑙𝑖=π‘˜β€–β€–π‘ π‘–βˆ’1β€–β€–β‹…π‘’π‘˜βˆ’1+𝑙𝑖=π‘˜β€–β€–π‘ π‘–βˆ’1β€–β€–ξ€·π‘’π‘–βˆ’1βˆ’π‘’π‘˜βˆ’1ξ€Έ,(3.20) where π‘ π‘–βˆ’1=π‘₯π‘–βˆ’π‘₯π‘–βˆ’1, that is, 𝑙𝑖=π‘˜β€–β€–π‘ π‘–βˆ’1β€–β€–β‹…π‘’π‘˜βˆ’1=ξ€·π‘₯π‘™βˆ’π‘₯π‘˜βˆ’1ξ€Έβˆ’π‘™ξ“π‘–=π‘˜β€–β€–π‘ π‘–βˆ’1β€–β€–ξ€·π‘’π‘–βˆ’1βˆ’π‘’π‘˜βˆ’1ξ€Έ.(3.21) From Assumption H, we know that there exists a constant πœ‰>0 such that β€–π‘₯β€–β‰€πœ‰,forπ‘₯βˆˆπ‘‰.(3.22) From (3.21) and the above inequality, one has 𝑙𝑖=π‘˜β€–β€–π‘ π‘–βˆ’1‖‖≀2πœ‰+𝑙𝑖=π‘˜β€–β€–π‘ π‘–βˆ’1β€–β€–β‹…β€–β€–π‘’π‘–βˆ’1βˆ’π‘’π‘˜βˆ’1β€–β€–.(3.23) Let Ξ” be a positive integer and Ξ”βˆˆ[8πœ‰/πœ†,8πœ‰/πœ†+1) where πœ† has been defined in Lemma 3.4. From Lemma 3.2, we know that there exists π‘˜0 such that 𝑖β‰₯π‘˜0‖‖𝑒𝑖+1βˆ’π‘’π‘–β€–β€–2≀14Ξ”.(3.24) From the Cauchy-Schwartz inequality and (3.24), forallπ‘–βˆˆ[π‘˜,π‘˜+Ξ”βˆ’1], one has β€–β€–π‘’π‘–βˆ’1βˆ’π‘’π‘˜βˆ’1β€–β€–β‰€π‘–βˆ’1𝑗=π‘˜β€–β€–π‘’π‘—βˆ’π‘’π‘—βˆ’1‖‖≀(π‘–βˆ’π‘˜)1/2ξƒ©π‘–βˆ’1𝑗=π‘˜β€–β€–π‘’π‘—βˆ’π‘’π‘—βˆ’1β€–β€–2ξƒͺ1/2≀Δ1/2β‹…ξ‚€14Ξ”1/2=12.(3.25) By Lemma 3.4, we know that there exists π‘˜β‰₯π‘˜0 such that ||β„œπœ†π‘˜,Ξ”||>Ξ”2.(3.26) It follows from (3.23), (3.25), and (3.26) that πœ†Ξ”4<πœ†2||β„œπœ†π‘˜,Ξ”||<12π‘˜+Ξ”βˆ’1𝑖=π‘˜β€–β€–π‘ π‘–βˆ’1‖‖≀2πœ‰.(3.27) From (3.27), one has Ξ”<8πœ‰/πœ†, which is a contradiction with the definition of Ξ”. Hence, liminfπ‘˜β†’+βˆžβ€–β€–π‘”π‘˜β€–β€–=0,(3.28) which completes the proof.

4. Numerical Results

In this section, we compare the modified PRP conjugate gradient method, denoted the MPRP method, to VPRP method, CG-DESCENT method, and DL+ method under the strong Wolfe line search about problems [20] with the given initial points and dimensions. The parameters are chosen as follows: 𝛿=0.01, 𝜎=0.1, 𝑣=1.25, πœ‚=0.01, and 𝑑=0.1. If ||π‘”π‘˜||≀10βˆ’6 is satisfied, we will stop the program. The program will be also stopped if the number of iteration is more than ten thousands. All codes were written in Matlab 7.0 and run on a PC with 2.0 GHz CPU processor and 512 MB memory and Windows XP operation system.

The numerical results of our tests with respect to the MPRP method, VPRP method, CG-DESCENT method, and DL+ method are reported in Tables 1, 2, 3, 4, respectively. In the tables, the column β€œProblem” represents the problem’s name in [20], and β€œCPU,” β€œNI,” β€œNF,” and β€œNG” denote the CPU time in seconds, the number of iterations, function evaluations, gradient evaluations, respectively. β€œDim” denotes the dimension of the tested problem. If the limit of iteration was exceeded, the run was stopped, and this is indicated by NaN.

Table 1 
The numerical results of the modified PRP method.
Table 2 
The numerical results of the VPRP method.
Table 3 
The numerical results of the CG-DESCENT method.
Table 4 
The numerical results of the DL+ method.

In this paper, we will adopt the performance profiles by Dolan and MorΓ© [21] to compare the MPRP method to the VPRP method, CG-DESCENT method, and DL+ method in the CPU time, the number of iterations, function evaluations, and gradient evaluations performance, respectively (see Figures 1, 2, 3, 4). In figures, 1𝑋=πœβŸΌπ‘›π‘ξ€½sizeπ‘βˆˆπ‘ƒβˆΆlog2ξ€·π‘Ÿπ‘,π‘ ξ€Έξ€Ύξ€½π‘Ÿβ‰€πœ,π‘Œ=𝑃𝑝,π‘ β‰€πœβˆΆ1≀𝑠≀𝑛𝑠.(4.1)


Figure 1 
Performance profiles of the conjugate gradient methods with respect to CPU time.

Figures 1–4 show the performance of the four methods relative to CPU time, the number of iterations, the number of function evaluations, and the number of gradient evaluations, respectively. For example, the performance profiles with respect to CPU time means that for each method, we plot the fraction 𝑃 of problems for which the method is within a factor 𝜏 of the best time. The left side of the figure gives the percentage of the test problems for which a method is the fastest; the right side gives the percentage of the test problems that are successfully solved by each of the methods. The top curve is the method that solved of the most problems in a time that was within a factor 𝜏 of the best time.


Figure 2 
Performance profiles with respect to the number of iterations.

Figure 3 
Performance profiles with respect to the number of function evaluations.

Figure 4 
Performance profiles with respect to the number of gradient evaluations.

Obviously, Figure 1 shows that MPRP method outperforms VPRP method, CG-DESCENT method, and DL+ method for the given test problems in the CPU time. Figures 2–4 show that the MPRP method also has the best performance with respect to the number of iterations and function and gradient evaluations since it corresponds to the top curve. So, the MPRP method is computationally efficient.

5. Conclusions

We have proposed a modified PRP method on the basic of the PRP method, which can generate sufficient descent directions with inexact line search. Moreover, we proved that the proposed modified method converge globally for general nonconvex functions. The performance profiles showed that the proposed method is also very efficient.

Acknowledgments

The authors wish to express their heartfelt thanks to the referees and Professor Piermarco Cannarsa for their detailed and helpful suggestions for revising the paper. This work was supported by The Nature Science Foundation of Chongqing Education Committee (KJ091104) and Chongqing Three Gorge University (09ZZ-060).