About this Journal Submit a Manuscript Table of Contents
Journal of Applied Mathematics
Volume 2012 (2012), Article ID 875494, 11 pages
doi:10.1155/2012/875494
Research Article

Accumulative Approach in Multistep Diagonal Gradient-Type Method for Large-Scale Unconstrained Optimization

1Department of Mathematics, University Putra Malaysia, Selangor, 43400 Serdang, Malaysia
2School of Computing and Maths, Charles Sturt University, Mitchell, NSW 2795, Australia

Received 2 April 2012; Revised 10 May 2012; Accepted 16 May 2012

Academic Editor: Vu Phat

Copyright © 2012 Mahboubeh Farid et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This paper focuses on developing diagonal gradient-type methods that employ accumulative approach in multistep diagonal updating to determine a better Hessian approximation in each step. The interpolating curve is used to derive a generalization of the weak secant equation, which will carry the information of the local Hessian. The new parameterization of the interpolating curve in variable space is obtained by utilizing accumulative approach via a norm weighting defined by two positive definite weighting matrices. We also note that the storage needed for all computation of the proposed method is just 𝑂(𝑛). Numerical results show that the proposed algorithm is efficient and superior by comparison with some other gradient-type methods.

1. Introduction

Consider the unconstrained optimization problem: min𝑓(𝑥),𝑥𝑅𝑛,(1.1) where 𝑓𝑅𝑛𝑅 is twice continuously differentiable function. The gradient-type methods for solving (1.1) can be written as 𝑥𝑘+1=𝑥𝑘𝐵𝑘1𝑔𝑘,(1.2) where 𝑔𝑘 and 𝐵𝑘 denote the gradient and the Hessian approximation of 𝑓 at 𝑥𝑘, respectively. By considering 𝐵𝑘=𝛼𝑘𝐼, Barzilai and Borwein (BB) [1] give 𝛼𝑘+1=𝑠𝑇𝑘𝑦𝑘𝑠𝑇𝑘𝑠𝑘,(1.3) where it is derived by minimizing 𝛼𝑘+1𝑠𝑘𝑦𝑘2 respect to 𝛼 with 𝑠𝑘=𝑥𝑘+1𝑥𝑘 and 𝑦𝑘=𝑔𝑘+1𝑔𝑘. Recently, some improved one-step gradient-type methods [25] in the frame of BB algorithm were proposed to solve (1.1). It is proposed to let 𝐵𝑘 be a diagonal nonsingular approximation to the Hessian and a new approximating matrix 𝐵𝑘+1 to the Hessian is developed based on weak secant equation of Dennis and Wolkowicz [6] 𝑠𝑇𝑘𝐵𝑘+1𝑠𝑘=𝑠𝑇𝑘𝑦𝑘.(1.4) In one-step method, data from one previous step is used to revise the current approximation of Hessian. Later Farid and Leong [7, 8] proposed multistep diagonal gradient methods inspired by the multistep quasi-Newton method of Ford [9, 10]. In this multistep framework, a fixed-point approach for interpolating polynomials was derived from data in previous iterations (not only one previous step) [710]. General approach of multistep method is based on the measurement of distances in the variable space where the distance of every iterate is measured from one-selected iterate. In this paper, we are interested to develop multistep diagonal updating based on accumulative approach for defining new parameter value of interpolating curve. From this point, the distance is accumulated between consecutive iterates as they are traversed in the natural sequence. For measuring the distance, we need to parameterize the interpolating polynomial through a norm that is defined by a positive definite weighting matrix, say 𝑀. Therefore, the performance of the multistep method may be significantly improved by carefully defining the weighting matrix. The rest of paper is organized as follows. In Section 2, we discuss a new multistep diagonal updating scheme based on the accumulative approach. In Section 3, we establish the global convergence of our proposed method. Section 4 presents numerical result and comparisons with BB method and one-step diagonal gradient method are reported. Conclusions are given in Section 5.

2. Derivation of the New Diagonal Updating via Accumulative Approach

This section motivates to state new implicit updates for diagonal gradient-type method through accumulative approach to determining a better Hessian approximation at each iteration. In multistep diagonal updating methods, weak secant equation (1.4) may be generalized by means of interpolating polynomials, instead of employing data just from one previous iteration like in one-step methods. Our aim is to derive efficient strategies for choosing a suitable set of parameters to construct the interpolating curve and investigate the best norm for measurement of the distances required to parameterize the interpolating polynomials. In general, this method obeys the recursive formula of the form 𝑥𝑘+1=𝑥𝑘𝛼𝑘𝐵𝑘1𝑔𝑘,(2.1) where 𝑥𝑘 is the 𝑘th iteration point, 𝛼𝑘 is step length which is determined by a line search, 𝐵𝑘 is an approximation to the Hessian in a diagonal form, and 𝑔𝑘 is the gradient of 𝑓 at 𝑥𝑘. Consider a differentiable curve 𝑥(𝜏) in 𝑅𝑛. The derivative of 𝑔(𝑥(𝜏)), at point 𝑥(𝜏), can be obtained by applying the chain rule: 𝑑𝑔|||𝑑𝜏𝜏=𝜏=𝐺(𝑥(𝜏))𝑑𝑥|||𝑑𝜏𝜏=𝜏.(2.2) We are interested to derive a relation that will be satisfied by the approximation of Hessian in diagonal form at 𝑥𝑘+1. If we assume that 𝑥(𝜏) passes through 𝑥𝑘+1 and choose 𝜏 so that 𝑥𝜏=𝑥𝑘+1,(2.3) then we have 𝐺𝑥𝑘+1𝑑𝑥𝑘+1=𝑥𝑑𝜏𝑑𝑔𝑘+1.𝑑𝜏(2.4) As in this paper, we use two-step method, therefore; we use information of most recent points 𝑥𝑘1, 𝑥𝑘, 𝑥𝑘+1 and their associated gradients. Consider 𝑥(𝜏) as the interpolating vector polynomial of degree 2: 𝑥𝜏𝑗=𝑥𝑘+𝑗1𝑗=0,1,2.(2.5) The selection of distinct scalar value 𝜏𝑗 efficiently through the new approach is the main contribution of this paper and will be discussed later in this section. Let (𝜏) be the interpolation for approximating the gradient vector: 𝜏𝑗𝑥=𝑔𝑘+𝑗1𝑗=0,1,2.(2.6) By denoting 𝑥(𝜏2)=𝑥𝑘+1 and defining 𝜏𝑑𝑥2𝑑𝜏=𝑟𝑘,𝑥𝜏𝑑𝑔2𝑑𝜏=𝑤𝑘,(2.7) we can obtain our desired relation that will be satisfied by the Hessian approximation at 𝑥𝑘+1 in diagonal form. Corresponding to this two-step approach, weak secant equation will be generalized as follows: 𝑟𝑇𝑘𝐵𝑘+1𝑟𝑘=𝑟𝑇𝑘𝑤𝑘.(2.8) Then, 𝐵𝑘+1 can be obtained by using an appropriately modified version of diagonal updating formula in [3] as follows: 𝐵𝑘+1=𝐵𝑘+𝑟𝑇𝑘𝑤𝑘𝑟𝑇𝑘𝐵𝑘𝑟𝑘𝐹tr2𝑘𝐹𝑘,(2.9) where 𝐹𝑘=diag((𝑟𝑘(1))2,(𝑟𝑘(2))2,,(𝑟𝑘(𝑛))2). Now, we attempt to construct an algorithm for finding desired vector 𝑟𝑘 and 𝑤𝑘 to improve the Hessian approximation. The proposed method is outlined as follows. First, we seek to derive strategies for choosing a suitable set of values 𝜏0, 𝜏1, and 𝜏2. The choice of {𝜏𝑗}2𝑗=0 is such that to reflect distances between iterates 𝑥𝑘 in 𝑅𝑛 that are dependent on some metric of the following general form: 𝜙𝑀𝑧1,𝑧2=(𝑧1𝑧2)𝑇𝑀(𝑧1𝑧2)1/2.(2.10) The establishment on 𝜏𝑗 can be made via the so-called accumulative approach where the accumulating distances (measured by the metric 𝜙𝑀) between consecutive iterates are used to approximate 𝜏𝑗. This leads to the following definitions (where without loss of generality, we take 𝜏1 to be origin for value of 𝜏): 𝜏1𝜏=0,𝑗=𝜏𝑗+1𝜙𝑀𝑥𝑘+𝑗,𝑥𝑘+𝑗1𝑗=0,2.(2.11) Then, we can construct the set {𝜏𝑗}2𝑗=0 as follows: 𝜏0=𝜏1𝜙𝑀𝑥𝑘,𝑥𝑘1def𝑥=𝑘𝑥𝑘1𝑀𝑠=𝑘1𝑀,𝜏2=𝜙𝑀𝑥𝑘+1,𝑥𝑘def=𝑥𝑘+1𝑥𝑘𝑀=𝑠𝑘𝑀,(2.12) where 𝑟𝑘 and 𝑤𝑘 are depending on the value of 𝜏. As the set {𝜏𝑗}2𝑗=0 measures the distances, therefore they need to be parameterized the interpolating polynomials via a norm defined by a positive definite matrix 𝑀. It is necessary to choose 𝑀 with some care, while improving the approximation of Hessian can be strongly influenced via the choice of 𝑀. Two choices for the weighting matrix 𝑀 are considered in this paper. In first choice, if 𝑀=𝐼, the 𝑀 reduces to the Euclidean norm, and then we obtain the following 𝜏𝑗 values accordingly: 𝜏2=𝑠𝑘2,𝜏1=0,𝜏0𝑠=𝑘12.(2.13) The second choice of weighting matrix 𝑀 is to take 𝑀=𝐵𝑘, where the current 𝐵𝑘 is diagonal approximation to the Hessian. By these two means, the measurement of the relevant distances is determined by the properties of the current quadratic approximation (based on 𝐵𝑘) to the objective function: 𝜏2=𝑠𝑘𝐵𝑘s𝑘1/2,𝜏1=0,𝜏0𝑠=𝑘1𝐵𝑘𝑠𝑘11/2.(2.14) Since 𝐵𝑘 is a diagonal matrix, then it is not expensive to compute {𝜏𝑗}2𝑗=0 at each iteration. The quantity 𝛿 is introduced here and defined as follows: 𝛿def=𝜏2𝜏1𝜏1𝜏0,(2.15) and 𝑟𝑘 and 𝑤𝑘 are given by the following expressions: 𝑟𝑘=𝑠𝑘𝛿2𝑠1+2𝛿𝑘1,𝑤(2.16)𝑘=𝑦𝑘𝛿2𝑦1+2𝛿𝑘1.(2.17) To safeguard on the possibility of having very small or very large 𝑟𝑇𝑘𝑤𝑘, we require that the condition 𝜀1𝑟𝑘22𝑟𝑇𝑘𝑤𝑘𝜀2𝑟𝑘22(2.18) is satisfied (we use 𝜀1=106 and 𝜀2=106). If not, then we replace 𝑟𝑘=𝑠𝑘 and 𝑤𝑘=𝑦𝑘. More that the Hessian approximation (𝐵𝑘+1) might not preserve the positive definiteness in each step. One of the fundamental concepts in this paper is to determine an “improved” version of the Hessian approximation to be used even in computing the metric when 𝑀=𝐵𝑘 and a weighing matrix as norm should be positive definite. To ensure that the updates remain positive definite, a scaling strategy proposed in [7] is applied. Hence, the new updating formula that incorporates the scaling strategy is given by 𝐵𝑘+1=𝜂𝑘𝐵𝑘+𝑟𝑇𝑘𝑤𝑘𝜂𝑘𝑟𝑇𝑘𝐵𝑘𝑟𝑘𝐹tr2𝑘𝐹𝑘,(2.19) where 𝜂𝑘𝑟=min𝑇𝑘𝑤𝑘𝑟𝑇𝑘𝐵𝑘𝑟𝑘,1.(2.20) This guarantees that the updated Hessian approximation is positive. Finally, the new accumulative MD algorithm is outlined as follows.

2.1. Accumulative MD Algorithm

Step 1. Choose an initial point 𝑥0𝑅𝑛, and a positive definite matrix 𝐵0=𝐼.
Let 𝑘=0.

Step 2. Compute 𝑔𝑘. If 𝑔𝑘𝜖, stop.

Step 3. If 𝑘=0, set 𝑥1=𝑥0(𝑔0/𝑔0). If 𝑘=1 set 𝑟𝑘=𝑠𝑘 and 𝑤𝑘=𝑦𝑘 go to Step 5.

Step 4. If 𝑘2 and 𝑀=𝐼 is considered, compute {𝜏𝑗}2𝑗=0 from (2.13).
Else if 𝑀=𝐵𝑘, compute {𝜏𝑗}2𝑗=0 from (2.14).
Compute 𝛿𝑘, 𝑟𝑘,𝑤𝑘 and 𝜂𝑘, from (2.15), (2.16),  (2.17), and (2.20), respectively.
If 𝑟𝑇𝑘𝑤𝑘104𝑟𝑘2𝑤𝑘2, set 𝑟𝑘=𝑠𝑘 and 𝑤k=𝑦𝑘.

Step 5. Compute 𝑑𝑘=𝐵𝑘1𝑔𝑘 and calculate 𝛼𝑘>0 such that the Armijo [11], condition holds:
𝑓(𝑥𝑘+1)𝑓(𝑥𝑘)+𝜎𝛼𝑘𝑔𝑇𝑘𝑑𝑘, where 𝜎(0,1) is a given constant.

Step 6. Let 𝑥𝑘+1=𝑥𝑘𝛼𝑘𝐵𝑘1𝑔𝑘, and update 𝐵𝑘+1 by (2.19).

Step 7. Set 𝑘=𝑘+1, and return to Step 2.

3. Convergence Analysis

This section is devoted to study the convergence of accumulative MD algorithm, when applied to the minimization of a convex function. To begin, we give the following result, which is due to Byrd and Nocedal [12] for the step generated by the Armijo line search algorithm. Here and elsewhere, denotes the Euclidean norm.

Theorem 3.1. Assume that 𝑓 is a strictly convex function. Suppose the Armijo line search algorithm is employed in a way that for any 𝑑𝑘 with 𝑑𝑇𝑘𝑔𝑘<0, the step length, 𝛼𝑘 satisfies 𝑓𝑥𝑘+𝛼𝑘𝑑𝑘𝑥𝑓𝑘+𝜎𝛼𝑘𝑔𝑇𝑘𝑑𝑘,(3.1) where 𝛼𝑘[𝜏,𝜏], 0<𝜏<𝜏 and 𝜎(0,1). Then, there exist positive constants 𝜌1 and 𝜌2 such that either 𝑓𝑥𝑘+𝛼𝑘𝑑𝑘𝑥𝑓𝑘𝜌1𝑑𝑇𝑘𝑔𝑘2𝑑𝑘2(3.2) or 𝑓𝑥𝑘+𝛼𝑘𝑑𝑘𝑥𝑓𝑘𝜌2𝑑𝑇𝑘𝑔𝑘(3.3) is satisfied.

We can apply Theorem 3.1 to establish the convergence of some Armijo-type line search methods.

Theorem 3.2. Assume that 𝑓 is a strictly convex function. Suppose that the Armijo line search algorithm in Theorem 3.1 is employed with 𝑑𝑘 chosen to obey the following conditions: there exist positive constants 𝑐1 and 𝑐2 such that 𝑔𝑇𝑘𝑑𝑘𝑐1𝑔𝑘2,𝑑𝑘𝑐2𝑔𝑘,(3.4) for all sufficiently large 𝑘. Then, the iterates 𝑥𝑘 generated by the line search algorithm have the property that liminf𝑘𝑔𝑘=0.(3.5)

Proof. By (3.4), we have that either (3.2) or (3.6) becomes 𝑓𝑥𝑘+𝛼𝑘𝑑𝑘𝑥𝑓𝑘𝑔𝑐𝑘2,(3.6) for some positive constants. Since 𝑓 is strictly convex, it is also bounded below. Then, (3.1) implies that 𝑓(𝑥𝑘+𝛼𝑘𝑑𝑘)𝑓(𝑥𝑘)0 as 𝑘. This also implies that 𝑔𝑘0 as 𝑘 or at least liminf𝑘𝑔𝑘=0.(3.7)

To prove that the accumulative MD algorithm is globally convergent when applied to the minimization of a convex function, it is sufficient to show that the sequence {𝐵𝑘} generated by (2.19)-(2.20) is bounded both above and below, for all finite 𝑘 so that its associated search direction satisfies condition (3.4). Since 𝐵𝑘 is diagonal, it is enough to show that each element of 𝐵𝑘 says 𝐵𝑘(𝑖);  𝑖=1,,𝑛 is bounded above and below by some positive constants. The following theorem gives the boundedness of {𝐵𝑘}.

Theorem 3.3. Assume that 𝑓 is strictly convex function where there exists positive constants 𝑚 and 𝑀 such that 𝑚𝑧2𝑧𝑇2𝑓(𝑥)𝑧𝑀𝑧2,(3.8) for all 𝑥,𝑧𝑅𝑛. Let {𝐵𝑘} be a sequence generated by the accumulative MD method. Then, 𝐵𝑘 is bounded above and below for all finite 𝑘, by some positive constants.

Proof. Let 𝐵𝑘(𝑖) be the 𝑖th element of 𝐵𝑘. Suppose 𝐵0 is chosen such that 𝜔1𝐵0(𝑖)𝜔2;𝑖=1,,𝑛, where 𝜔1 and 𝜔2 are some positive constants.
Case 1. If (2.18) is satisfied, we have 𝐵1=𝜂0𝐵0;if𝑟𝑇0𝑤0<𝑟𝑇0𝐵0𝑟0𝐵0+𝑟𝑇0𝑤0𝑟𝑇0𝐵0𝑟𝑘𝐹tr20𝐹0;if𝑟𝑇0𝑤0𝑟𝑇0𝐵0𝑟0.(3.9) By (2.18) and the definition of 𝜂𝑘, one can obtain 𝜀1𝜔2𝜂01.(3.10) Thus, if 𝑟𝑇0𝑤0<𝑟𝑇0𝐵0𝑟0, then 𝐵1=𝜂0𝐵0 satisfies 𝜀1𝜔1𝜔2𝐵1(𝑖)𝜔2.(3.11) On the other hand, if 𝑟𝑇0𝑤0𝑟𝑇0𝐵0𝑟0, then 𝐵1(𝑖)=𝐵0(𝑖)+𝑟𝑇0𝑤0𝑟𝑇0𝐵0𝑟0𝐹tr20𝑟0(𝑖)2,(3.12) where 𝑟0(𝑖) is the 𝑖th component of 𝑟0. Letting (𝑟0(𝑀)) be the largest component (in magnitude) of 𝑟0, that is, (𝑟0(𝑖))2(𝑟0(𝑀))2; for all 𝑖, then it follows that 𝑟02𝑛(𝑟0(𝑀))2, and the property of 𝑟𝑇0𝑤0𝑟𝑇0𝐵0𝑟0, (3.12) becomes 𝜔1𝐵0(𝑖)𝐵1(𝑖)𝜔2+𝑛𝜀2𝜔1𝐹tr20𝑟0(𝑀)4𝜔2𝜀+𝑛2𝜔1.(3.13) Hence, 𝐵1(𝑖) is bounded above and below, for all 𝑖 in both occasions.
Case 2. If (2.18) is violated, then the updating formula for 𝐵1 becomes 𝐵1(𝑖)=𝜂0𝐵0(𝑖)+𝑠𝑇0𝑦0𝜂0𝑠𝑇0𝐵0𝑠0𝐸tr20𝑠0(𝑖)2,(3.14) where 𝑠0(𝑖) is the 𝑖th component of 𝑠0, 𝐸0=diag((𝑠0(1))2,(𝑠0(2))2,,(𝑠0(𝑛))2), and 𝜂0=min(1,𝑠𝑇0𝑦0/𝑠𝑇0𝐵0𝑠0).
Because 𝜂01 also implies that 𝑠𝑇0𝑦0𝑠𝑇0𝐵0𝑠00, then this fact, together with the convexity property (3.8), and the definition of 𝜂 give 𝑚min1,𝜔2𝜔1𝜂0𝐵0(𝑖)𝐵1(𝑖)𝐵0(𝑖)+𝑀𝜔1𝑠02𝐸tr20𝑠0(𝑖)2.(3.15) Using the similar argument as above, that is, by letting 𝑠0(𝑀) be the largest component (in magnitude) of 𝑠0, then it follows that 𝑚min1,𝜔2𝜔1𝐵1(𝑖)𝜔2+𝑛𝑀𝜔1.(3.16)
Hence, in both cases, 𝐵1(𝑖) is bounded above and below, by some positive constants. Since the upper and lower bound for 𝐵1(𝑖) is, respectively, independent to 𝑘, we can proceed by using induction to show that 𝐵𝑘(𝑖) is bounded, for all finite 𝑘.

4. Numerical Results

In this section, we examine the practical performance of our proposed algorithm in comparison with the BB method and standard one-step diagonal gradient-type method (MD). The new algorithms are referred to as AMD1 and AMD2 when 𝑀=𝐼 and 𝑀=𝐵𝑘 are used, respectively. For all methods we employ Armijo line search [11] where 𝜎=0.9. All experiments in this paper are implemented on a PC with Core Duo CPU using Matlab 7.0. For each run, the termination condition is that 𝑔𝑘104. All attempts to solve the test problems were limited to a maximum of 1000 iterations. The test problems are chosen from Andrei [13] and Moré et al. [14] collections. The detailed test problem is summarized in Table 1. Our experiments are performed on a set of 36 nonlinear unconstrained problems, and the problems vary in size from 𝑛=10 to 10000 variables. Figures 1, 2, and 3 present the Dolan and Moré [15] performance profile for all algorithms subject to the iteration, function call, and CPU time.

tab1
Table 1: Test problem and its dimension.
875494.fig.001
Figure 1: Performance profile based on Iteration for all problems.
875494.fig.002
Figure 2: Performance profile based on function call.
875494.fig.003
Figure 3: Performance profile based on CPU time per iteration.

From Figure 1, we see that AMD2 method is the top performer, being more successful than other methods in the number of iteration. Figure 2 shows that AMD2 method requires the fewest function calls. From Figure 3, we observe that the AMD2 method is faster than MD and AMD1 methods and needs reasonable time to solve large-scale problems when compared to the BB method. At each iteration, the proposed method does not require more storage than classic diagonal updating methods. Moreover, a higher-order accuracy in approximating the Hessian matrix of the objective function makes AMD method need less iterations and less function evaluation. The numerical results by the tests reported in Figures 1, 2, and 3 demonstrate clearly the new method AMD2 shows significant improvements, when compared with BB, MD, and AMD1. Generally, 𝑀=𝐵𝑘 performs better than 𝑀=𝐼. It is most probably due to the fact that 𝐵𝑘 is a better Hessian approximation than the identity matrix 𝐼.

5. Conclusion

In this paper, we propose a new two-step diagonal gradient method as view of accumulative approach for unconstrained optimization. The new parameterization for multistep diagonal gradient-type method is developed via employing accumulative approach. The new technique is devised for interpolating curves which are the basis of multistep approach. Numerical results show that the proposed method is suitable to solve large-scale unconstrained optimization problems and more stable than other similar methods in practical computation. The improvement that our proposed methods bring does come at a complexity cost of 𝑂(𝑛) while others are about 𝑂(𝑛2) [9, 10].

References

  1. J. Barzilai and J. M. Borwein, “Two-point step size gradient methods,” IMA Journal of Numerical Analysis, vol. 8, no. 1, pp. 141–148, 1988. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  2. M. A. Hassan, W. J. Leong, and M. Farid, “A new gradient method via quasi-Cauchy relation which guarantees descent,” Journal of Computational and Applied Mathematics, vol. 230, no. 1, pp. 300–305, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  3. W. J. Leong, M. A. Hassan, and M. Farid, “A monotone gradient method via weak secant equation for unconstrained optimization,” Taiwanese Journal of Mathematics, vol. 14, no. 2, pp. 413–423, 2010. View at Zentralblatt MATH
  4. W. J. Leong, M. Farid, and M. A. Hassan, “Scaling on diagonal Quasi-Newton update for large scale unconstrained Optimization,” Bulletin of the Malaysian Mathematical Sciences Soceity, vol. 35, no. 2, pp. 247–256, 2012.
  5. M. Y. Waziri, W. J. Leong, M. A. Hassan, and M. Monsi, “A new Newtons method with diagonal jacobian approximation for systems of nonlinear equations,” Journal of Mathematics and Statistics, vol. 6, pp. 246–252, 2010. View at Publisher · View at Google Scholar
  6. J. E. Dennis, Jr. and H. Wolkowicz, “Sizing and least-change secant methods,” SIAM Journal on Numerical Analysis, vol. 30, no. 5, pp. 1291–1314, 1993. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  7. M. Farid, W. J. Leong, and M. A. Hassan, “A new two-step gradient-type method for large-scale unconstrained optimization,” Computers & Mathematics with Applications, vol. 59, no. 10, pp. 3301–3307, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  8. M. Farid and W. J. Leong, “An improved multi-step gradient-type method for large scale optimization,” Computers & Mathematics with Applications, vol. 61, no. 11, pp. 3312–3318, 2011. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  9. J. A. Ford and I. A. Moghrabi, “Alternating multi-step quasi-Newton methods for unconstrained optimization,” Journal of Computational and Applied Mathematics, vol. 82, no. 1-2, pp. 105–116, 1997. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  10. J. A. Ford and S. Tharmlikit, “New implicit updates in multi-step quasi-Newton methods for unconstrained optimisation,” Journal of Computational and Applied Mathematics, vol. 152, no. 1-2, pp. 133–146, 2003. View at Publisher · View at Google Scholar
  11. L. Armijo, “Minimization of functions having Lipschitz continuous first partial derivatives,” Pacific Journal of Mathematics, vol. 16, pp. 1–3, 1966. View at Zentralblatt MATH
  12. R. H. Byrd and J. Nocedal, “A tool for the analysis of quasi-Newton methods with application to unconstrained minimization,” SIAM Journal on Numerical Analysis, vol. 26, no. 3, pp. 727–739, 1989. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  13. N. Andrei, “An unconstrained optimization test functions collection,” Advanced Modeling and Optimization, vol. 10, no. 1, pp. 147–161, 2008. View at Zentralblatt MATH
  14. J. J. Moré, B. S. Garbow, and K. E. Hillstrom, “Testing unconstrained optimization software,” ACM Transactions on Mathematical Software, vol. 7, no. 1, pp. 17–41, 1981. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  15. E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming A, vol. 91, no. 2, pp. 201–213, 2002. View at Publisher · View at Google Scholar · View at Zentralblatt MATH