Special Issue

Machine Learning and its Applications in Image Restoration

View this Special Issue

Research Article | Open Access

Volume 2020 |Article ID 9280495 | https://doi.org/10.1155/2020/9280495

Pengyuan Li, Zhan Wang, Dan Luo, Hongtruong Pham, "Global Convergence of a Modified Two-Parameter Scaled BFGS Method with Yuan-Wei-Lu Line Search for Unconstrained Optimization", Mathematical Problems in Engineering, vol. 2020, Article ID 9280495, 15 pages, 2020. https://doi.org/10.1155/2020/9280495

Global Convergence of a Modified Two-Parameter Scaled BFGS Method with Yuan-Wei-Lu Line Search for Unconstrained Optimization

Guest Editor: Weijun Zhou
Accepted05 Aug 2020
Published26 Aug 2020

Abstract

The BFGS method is one of the most efficient quasi-Newton methods for solving small- and medium-size unconstrained optimization problems. For the sake of exploring its more interesting properties, a modified two-parameter scaled BFGS method is stated in this paper. The intention of the modified scaled BFGS method is to improve the eigenvalues structure of the BFGS update. In this method, the first two terms and the last term of the standard BFGS update formula are scaled with two different positive parameters, and the new value of is given. Meanwhile, Yuan-Wei-Lu line search is also proposed. Under the mentioned line search, the modified two-parameter scaled BFGS method is globally convergent for nonconvex functions. The extensive numerical experiments show that this form of the scaled BFGS method outperforms the standard BFGS method or some similar scaled methods.

1. Introduction

Considerwhere , and is a continuously differentiable function bounded from below. The quasi-Newton methods are currently used in countless optimization software for solving unconstrained optimization problems [18]. The BFGS method, one of the most efficient quasi-Newton methods, for solving (1) is an iterative method of the following form:where , obtained by some line search rule, is a step size, and is the BFGS search direction computed by the following equation:where is the gradient of , and the matrix is the BFGS approximation to the Hessian , which has the following update formula:where and . The problems related to the BFGS method have been analyzed and studied by many scholars, and satisfactory conclusions have been drawn [916]. In earlier year, Powell [17] first proved the global convergence of the standard BFGS method with inexact Wolfe line search for convex functions. Under the exact line search or some specific inexact line search, the BFGS method has the convergence property for convex minimization problems [1821]. By contrast, for nonconvex problems, Mascaren [22] has presented an example to elaborate that the BFGS method and some Broyden-type methods may not be convergent under the exact line search. As such, with the Wolfe line searches, Dai [23] also proved that the BFGS method may fail to converge. To verify the global convergence of the BFGS method for general functions and to obtain a better Hessian approximation matrix of the objective function, Yuan and Wei [24] presented a modified quasi-Newton equation as follows:where

In practice, the standard BFGS method has many qualities worth exploring and can effectively solve a class of unconstrained optimization problems.

Here, two excellent properties of the BFGS method are introduced. One is the self-correcting quality, scilicet; if the current Hessian approximate inverse matrix estimates the curvature of the function incorrectly, then Hessian approximation matrix will correct itself within a few steps. The other interesting property is that small eigenvalues are better corrected than large ones [25]. Hence, one can see that, the efficiency of the BFGS algorithm is subject to the eigenvalues structure of the Hessian approximation matrix intensely. To improve the performances of the BFGS method, Oren and Luenberger [26] scaled the Hessian approximation matrix , that is, they replaced by , where is a self-scaling factor. Nocedal and Yuan [27] further studied the self-scaling BFGS method when . Based on the value of this , Al-Baali [28] introduced a simple modification: . The numerical experiments showed that the modified self-scaling BFGS method outperforms the unscaled BFGS method. Many other scaled BFGS methods with better properties will be enumerated.

Formula 1. The general one-parameter scaled BFGS updating formula iswhere is a positive parameter, and it is diverse for the selection of the scaled factor , which is listed as follows.Choice A:where the value of is given by Yuan [29], and with inexact line search, the global convergence of the scaled BFGS method with given by (9) is established for convex functions by Powell [30]. Ulteriorly, for general nonlinear functions, Yuan limited the value range of to [0.01, 100] to ensure the positivity of under the inexact line search and proved the global convergence of the scaled BFGS method in this form.Choice B:which is obtained as a solution of the problem: . The scaled BFGS method based on this value of was introduced by Barzilai and Borwein [31] and was deemed the spectral scaled BFGS method. Cheng and Li [32] proved that the spectral scaled BFGS method is globally convergent under Wolfe line search with assuming the convexity of the minimizing function.Choice C:where for . Under the Wolfe line search (20) and (21), holds for , which implies that computed by (11) is bounded away from zero, that is to say, . Therefore, in this instance, the large eigenvalues of given by (8) are shifted to the left [33].

Formula 2. Proposed by Oren and Luenberger [26], this scaled BFGS method was the single parameter scaled of the first two items of the BFGS update and was defined aswhere is a positive parameter and is calculated as follows:

The parameter assigned by (13) can make the structure of eigenvalue to inverse Hessian approximation more easily analyzed. Consequently, it is regarded as one of the best factors.

Formula 3. In this method, the scaled parameters are selected to cluster the eigenvalues of the iteration matrix and shift the large eigenvalues to the left. The update formula of the Hessian approximate matrix is computed aswhere both and are positive parameters, and Andrei [34] preset them as the following values:

If the scaled parameters are bounded and line search is inexact, then this scaled BFGS algorithm is globally convergent for general functions. A large number of numerical experiments show that the double parameter scaled BFGS method with and given by (15) and (16) is more competitive than the standard BFGS method. In this paper, combining (7) and (14), we propose a new update formula of listed as follows:where is determined by formula (6),

Some interesting properties of the BFGS-type method are inseparable from the weak Wolfe–Powell (WWP) line search:where . There are many research studies based on this line search [3543]. To further develop the inexact line search, Yuan et al. present a new line search and call it Yuan-Wei-Lu (YWL) line search, which has the following form:where , , and . The main work of this paper is to verify the global convergence of the modified scaled BFGS update (17) with and given by (18) and (19), respectively, under this line search. Abundant numerical results show that such a combination is appropriate for nonconvex functions.

Our paper is organized as follows. The motivation and algorithm are introduced in the next section. In Section 3, the convergence analysis of the modified two-parameter scaled BFGS method under Yuan-Wei-Lu line search is established. Section 4 is devoted to show the results of numerical experiments. Some conclusions are stated in the last section.

2. Motivation and Algorithm

Two crucial tools for analyzing properties of the BFGS method are the trace and the determinant of the given by (4). Thus, the corresponding relations are enumerated as follows:

Applying the following existing relation in the study of Sun and Yuan [44],where , , , and ; we obtain

Obviously, the efficiency of the BFGS method depends on the eigenvalues structure of the Hessian approximation matrix, and the BFGS method is actually more affected by large eigenvalues than by small eigenvalues [25, 45, 46]. It can be seen that the second item on the right side of the formula (25) is negative. Therefore, it produces a shift of the eigenvalues of to the left. Thus, the BFGS method can modify large eigenvalues. Moreover, the third term on the right hand side of (25) being positive produces a shift of the eigenvalues of to the right. If this term is large, may have large eigenvalues too. Therefore, the eigenvalues of the can be corrected by scaling the corresponding items in (25), which is the main motivation for us to use the scaling BFGS method. In this paper, we scale the first two terms and the last term of the standard BFGS update formula with two different positive parameters and propose a new . In subsequent proof, we will propose some lemmas based on these two important tools to analyze the convergence of the modified scaled BFGS method. Then, an algorithm framework for solving the problem (1) will be built in Algorithm 1, which can be designed as

 Step 1: given an initial point , an symmetric positive definite matrix , is sufficiently small and choose constants , , and . Set . Step 2: if , stop. Step 3: obtain a search direction by solving Step 4: compute by Yuan-Wei-Lu line search conditions (22) and (23). Step 5: find the scaling factors and by (18) and (19). Step 6: let . Update by (17). Step 7: let , and go to Step 2.

3. Convergence Analysis

In Section 3, the global convergence of Algorithm 1 will be established, and the following assumptions are useful in convergence analysis.

Assumption 1. (i)The level set is bounded(ii)The function is twice continuously differentiable and bounded from below

Lemma 1. If is the positive definite, , and if is computed by (22) and (23), then given by (17) is an equally positive definite for all .

Proof. The inequality (22) and (23) indicates that . Using the definition of , we obtainFor any ,where the penultimate inequality follows, andwhich is obtained by the Cauchy–Schwarz inequality.

Lemma 2. Let be generated by (16) for , then and inclines to 1.

Proof. Observe the formula (19); after substituting , we can find that is close to 1. Owing to the symmetry, positive definiteness, and nonsingularity of , its eigenvalues is real and positive, and . Hence, for , and . Since , , and for sufficiently large , , and are roughly of the same order of magnitude, which shows that . To sum up, the relations and are valid, namely for , and inclines to 1. The proof is completed.

Remark 1. Based on the conclusion of lemma, we can infer that for any integer , there exist two positive constants satisfying .

Lemma 3. If is updated by (14), where and are determined by (18) and (16), then

Proof. Considering (25), we haveIn addition,Therefore, by Remark 1 and the above inequality, the formula (33) is transformed intowhich implies (31). From the positive definiteness of , (32) also holds. The proof is completed.

Lemma 4. Consider and for all , where and are constants. Then, there exists a positive constant such thatfor all sufficiently large.

Proof. Utilizing the identity (26) and taking the determinant on both sides of the formula (14) with and computed as in (18) and (16), we havewhere the penultimate inequality follows , , , and for all . Furthermore, by and Lemma 4, we obtainTherefore,Suppose is sufficiently large, (39) implies (36). The proof is completed.

Theorem 1. If the sequence is obtained by Algorithm 1, then

Proof. The proof by contradiction is used to prove (40) holds. Suppose that . By Yuan-Wei-Lu line search (22) and bounded below, we obtainAdding the abovementioned inequalities from to and utilizing Assumption 1 (ii), we haveFrom Assumption 1 (ii) and (42), we haveBased on this, given a constant , there is a positive integer satisfyingwhere is any positive integer, and the first inequality follows the geometric inequality. Moreover, by Lemma 4, we obtainConsidering , the above formula and formula (39) are contradictory. Thus, (40) is valid. The proof is completed.

4. Numerical Results

In this section, numerical results of Algorithm 1 are reported, and the following methods were compared: (i) MTPSBFGS method ( is updated by (17) with and given by (18) and (19)). (ii) SBFGS method ( is updated by (14) with and given by (11) and (16)).

4.1. General Unconstrained Optimisation Problems

Tested problems: a total of 74 test questions, listed in Table 1 and derived from the studies by Bongartz et al. and More et al. [47, 48].Parameters: Algorithm 1 runs with , , , , and .Dimensionality: the algorithm is tested in the following three dimensions: 300, 900, and 2700.Himmelblau stop rule [49]: if , then set or . The iterations are stopped if or holds, where and .Experiment environment: all programs are written in MATLAB R2014a and run on a PC with an Inter(R) Core(TM) i5-4210U CPU at 1.70 GHz, 8.00 GB of RAM, and the Windows 10 operating system.Symbol representation: No.: the test problem number. CPU time: the CPU time in seconds. NI: the number of iterations.  NFG: the total number of function and gradient evaluations.Image description: Figures 13 show the profiles for CPU time, NI, and NFG, and Tables 26 provide the detail numerical results. From these figures and tables, it is obvious that the MTPSBFGS method possesses better numerical performance between these two methods, that is, the proposed modified scaled BFGS method is reasonable and feasible. The specific reasons for good performance are stated as follows. The parameter scaling the first two terms of the standard BFGS update is determined to cluster the eigenvalues of this matrix, and the parameter scaling the third term is determined to reduce its large eigenvalues, thus obtaining a better distribution of them.

 No. Test problem 1 Extended Freudenstein and Roth function 2 Extended trigonometric function 3 Extended Rosenbrock function 4 Extended White and Holst function 5 Extended Beale function 6 Extended penalty function 7 Perturbed quadratic function 8 Raydan 1 function 9 Raydan 2 function 10 Diagonal 1 function 11 Diagonal 2 function 12 Diagonal 3 function 13 Hager function 14 Generalized tridiagonal 1 function 15 Extended tridiagonal 1 function 16 Extended three exponential terms function 17 Generalized tridiagonal 2 function 18 Diagonal 4 function 19 Diagonal 5 function 20 Extended Himmelblau function 21 Generalized PSC1 function 22 Extended PSC1 function 23 Extended Powell function 24 Extended block diagonal BD1 function 25 Extended Maratos function 26 Extended Cliff function 27 Quadratic diagonal perturbed function 28 Extended Wood function 29 Extended Hiebert function 30 Quadratic function QF1 function 31 Extended quadratic penalty QP1 function 32 Extended quadratic penalty QP2 function 33 A quadratic function QF2 function 34 Extended EP1 function 35 Extended tridiagonal 2 function 36 BDQRTIC function (CUTE) 37 TRIDIA function (CUTE) 38 ARWHEAD function (CUTE) 39 NONDIA function (CUTE) 40 NONDQUAR function (CUTE) 41 DQDRTIC function (CUTE) 42 EG2 function (CUTE) 43 DIXMAANA function (CUTE) 44 DIXMAANB function (CUTE) 45 DIXMAANC function (CUTE) 46 DIXMAANE function (CUTE) 47 Partial perturbed quadratic function 48 Broyden tridiagonal function 49 Almost perturbed quadratic function 50 Tridiagonal perturbed quadratic function 51 EDENSCH function (CUTE) 52 VARDIM function (CUTE) 53 STAIRCASE S1 function 54 LIARWHD function (CUTE) 55 DIAGONAL 6 function 56 DIXON3DQ function (CUTE) 57 DIXMAANF function (CUTE) 58 DIXMAANG function (CUTE) 59 DIXMAANH function (CUTE) 60 DIXMAANI function (CUTE) 61 DIXMAANJ function (CUTE) 62 DIXMAANK function (CUTE) 63 DIXMAANL function (CUTE) 64 DIXMAAND function (CUTE) 65 ENGVAL1 function (CUTE) 66 FLETCHCR function (CUTE) 67 COSINE function (CUTE) 68 Extended DENSCHNB function (CUTE) 69 Extended DENSCHNF function (CUTE) 70 SINQUAD function (CUTE) 71 BIGGSB1 function (CUTE) 72 Partial perturbed quadratic PPQ2 function 73 Scaled quadratic SQ1 function 74 Scaled quadratic SQ2 function
 MTPSBFGS-YWL SBFGS-WWP No. Dim NI NFG CPU time NI NFG CPU time 1 300 29 63 0.3125 25 57 0.3125 1 900 24 56 5.859375 23 54 5.96875 1 2700 22 48 86.9375 29 67 135.6875 2 300 50 114 0.59375 46 102 0.5 2 900 51 114 12.890625 50 112 13.515625 2 2700 53 120 217.90625 53 122 254.75 3 300 50 137 0.515625 47 120 0.5 3 900 75 215 17.59375 63 167 16.75 3 2700 43 135 184.5625 60 166 280.484375 4 300 104 370 1.25 62 176 0.75 4 900 87 258 22.453125 38 109 9.1875 4 2700 67 179 308.3125 50 149 236.984375 5 300 18 48 0.21875 22 58 0.296875 5 900 18 45 4.5625 20 46 5.171875 5 2700 20 45 90.25 23 58 106.984375 6 300 68 152 0.828125 68 152 0.796875 6 900 69 158 18.375 69 158 18.40625 6 2700 85 192 405.84375 85 192 410.765625 7 300 76 154 0.8125 76 154 0.875 7 900 133 268 36.6875 133 268 36.84375 7 2700 232 466 1158.734375 231 464 1151.625 8 300 22 49 0.203125 26 54 0.265625 8 900 25 55 6.515625 25 55 6.421875 8 2700 25 55 118.640625 25 55 116.859375 9 300 7 16 0.0625 12 26 0.109375 9 900 7 16 1.546875 12 26 2.84375 9 2700 8 18 31.96875 12 26 49.046875 10 300 2 9 0 2 9 0 10 900 2 9 0.0625 2 9 0.0625 10 2700 2 9 0.25 2 9 0.25 11 300 75 194 0.921875 46 94 0.59375 11 900 95 272 26.359375 66 134 18.484375 11 2700 6 20 19.875 97 196 472.390625 12 300 11 24 0.125 11 24 0.109375 12 900 13 28 3.453125 13 28 3.296875 12 2700 13 28 60.34375 13 28 58.40625 13 300 11 25 0.125 10 23 0.125 13 900 8 23 1.921875 10 26 2.46875 13 2700 19 94 88.65625 19 96 88.71875 14 300 8 20 0.125 8 20 0.15625 14 900 7 18 1.703125 7 18 1.703125 14 2700 7 18 27.6875 7 18 27.421875 15 300 19 43 0.359375 22 49 0.40625 15 900 25 57 7.125 39 82 11.078125 15 2700 25 55 119.40625 41 86 198.203125 16 300 9 21 0.109375 10 23 0.0625 16 900 8 18 1.921875 9 21 2.15625 16 2700 11 24 49.359375 10 22 43.046875 17 300 33 73 0.421875 32 75 0.390625 17 900 23 49 5.984375 23 49 5.8125 17 2700 25 53 115.890625 25 53 112.984375
 MTPSBFGS-YWL SBFGS-WWP No. Dim NI NFG CPU time NI NFG CPU time 18 300 3 10 0 3 10 0 18 900 3 10 0.53125 3 10 0.5 18 2700 3 10 9.09375 3 10 8.75 19 300 3 10 0 3 10 0.0625 19 900 3 10 0.53125 3 10 0.484375 19 2700 3 10 8.578125 3 10 8.421875 20 300 33 71 0.34375 32 69 0.359375 20 900 12 31 2.234375 12 31 2.171875 20 2700 12 35 44.546875 43 93 191.3125 21 300 25 56 0.28125 34 74 0.390625 21 900 25 56 6.703125 35 76 9.265625 21 2700 26 58 124.484375 36 78 170.34375 22 300 8 30 0.109375 8 31 0.125 22 900 8 31 1.90625 8 31 2 22 2700 8 31 33.515625 8 31 33.359375 23 300 34 85 0.359375 52 109 0.5625 23 900 39 89 10.65625 51 117 13.6875 23 2700 47 111 233.65625 47 112 223.71875 24 300 33 162 0.296875 29 145 0.234375 24 900 14 111 1.03125 37 171 6.890625 24 2700 14 111 15.34375 15 114 24.234375 25 300 90 262 1.0625 127 348 1.5 25 900 123 346 33.765625 88 257 23.703125 25 2700 56 139 270.703125 97 284 463.609375 26 300 56 134 0.640625 56 134 0.609375 26 900 65 152 17.359375 65 152 17.296875 26 2700 61 146 292.546875 61 146 287.25 27 300 6 16 0.0625 6 16 0 27 900 11 31 2.75 11 31 2.65625 27 2700 16 45 75.3125 17 44 79.671875 28 300 28 63 0.3125 27 61 0.296875 28 900 28 63 7.34375 27 62 6.921875 28 2700 25 61 119.5 25 61 117.265625 29 300 4 17 0.0625 4 15 0.046875 29 900 4 17 0.6875 4 17 0.828125 29 2700 4 17 9.484375 4 17 9.53125 30 300 93 188 1.203125 93 188 1.046875 30 900 164 330 45.96875 166 334 46.890625 30 2700 295 592 1476.90625 294 590 1474.96875 31 300 23 52 0.21875 23 52 0.21875 31 900 27 62 6.828125 27 62 6.84375 31 2700 28 64 126.734375 28 64 126.421875 32 300 22 46 0.28125 22 46 0.25 32 900 20 44 5.171875 20 44 5.03125 32 2700 47 98 219.515625 47 98 218.78125 33 300 5 11 0.0625 5 11 0.0625 33 900 4 9 0.625 4 9 0.59375 33 2700 3 7 7.421875 3 7 7.34375 34 300 3 7 0.0625 3 7 0 34 900 3 7 0.53125 3 7 0.5 34 2700 4 8 13.203125 4 8 13.234375
 MTPSBFGS-YWL SBFGS-WWP No. Dim NI NFG CPU time NI NFG CPU time 35 300 4 8 0 4 8 0 35 900 7 14 1.59375 7 14 1.59375 35 2700 11 22 46.875 11 22 46.40625 36 300 24 59 0.359375 26 64 0.46875 36 900 23 66 6.359375 20 58 5.59375 36 2700 18 49 82.359375 19 46 85.5625 37 300 136 275 1.65625 136 275 1.6875 37 900 235 473 67.40625 235 473 67.359375 37 2700 441 886 2240.015625 442 888 2234.09375 38 300 12 27 0.125 12 27 0.0625 38 900 12 26 2.8125 12 26 2.921875 38 2700 15 33 64.90625 15 33 64.65625 39 300 37 80 0.40625 37 80 0.46875 39 900 43 89 10.890625 43 91 11.265625 39 2700 26 52 116.703125 26 52 116.078125 40 300 532 1329 7.40625 958 1925 12.96875 40 900 546 1364 150.921875 1000 2008 274.5625 40 2700 644 1605 3124.65625 1000 2014 4810.296875 41 300 18 41 0.1875 20 45 0.1875 41 900 19 43 4.828125 19 43 4.734375 41 2700 19 43 85.0625 19 43 84.875 42 300 19 65 0.15625 16 57 0.078125 42 900 4 21 0.046875 4 21 0.046875 42 2700 4 21 0.578125 4 21 0.515625 43 300 22 48 0.265625 25 54 0.234375 43 900 23 50 5.84375 27 58 6.984375 43 2700 25 54 114.328125 29 62 133.78125 44 300 38 80 0.453125 39 82 0.546875 44 900 36 76 9.703125 43 90 11.5 44 2700 39 82 180.453125 46 96 212.125 45 300 17 40 0.171875 18 42 0.28125 45 900 17 40 4.265625 18 42 4.65625 45 2700 18 42 80.171875 19 44 85.09375 46 300 104 255 1.5625 60 126 0.875 46 900 141 354 41.203125 87 180 24.796875 46 2700 164 418 829.3125 116 238 586.296875 47 300 37 80 0.734375 36 78 0.71875 47 900 44 95 14.640625 44 95 14.78125 47 2700 13 31 64.21875 13 31 63.9375 48 300 26 52 0.296875 26 52 0.3125 48 900 48 96 12.890625 48 96 12.890625 48 2700 22 47 99.546875 22 47 99.96875 49 300 76 154 0.953125 76 154 0.859375 49 900 133 268 37.203125 134 270 37.59375 49 2700 232 466 1161.109375 234 470 1176.921875 50 300 76 154 1.234375 75 152 1.328125 50 900 132 266 38.859375 132 266 38.53125 50 2700 231 464 1165.1875 232 466 1170 51 300 23 48 0.296875 23 48 0.3125 51 900 23 48 5.890625 23 48 5.859375 51 2700 23 48 102.484375 23 48 103.453125
 MTPSBFGS-YWL SBFGS-WWP No. Dim NI NFG CPU time NI NFG CPU time 52 300 87 200 0.875 87 200 1.0625 52 900 103 236 27.6875 103 236 27.875 52 2700 118 270 566.65625 118 270 567.828125 53 300 938 2375 12.09375 388 778 4.671875 53 900 1000 2537 282.375 1000 2002 280.203125 53 2700 1000 2539 5006.875 1000 2002 5001.984375 54 300 31 72 0.328125 35 88 0.390625 54 900 42 107 10.8125 24 65 6.09375 54 2700 23 67 102.75 27 69 121.75 55 300 9 20 0.125 18 38 0.265625 55 900 10 22 2.234375 20 42 4.921875 55 2700 11 24 46.3125 21 44 92.390625 56 300 1000 2607 13.03125 375 750 4.5625 56 900 1000 2540 268.3125 1000 2000 277.515625 56 2700 1000 2539 4518.96875 1000 2000 4637.546875 57 300 107 260 1.578125 59 124 0.796875 57 900 99 236 28.140625 85 176 24.15625 57 2700 63 149 313.140625 105 216 522.734375 58 300 97 233 1.3125 69 144 0.890625 58 900 120 292 33.84375 99 204 27.828125 58 2700 144 344 717.328125 117 240 583.078125 59 300 24 73 0.25 76 169 0.890625 59 900 27 72 7.296875 27 71 7.296875 59 2700 31 78 148.625 79 173 389.4375 60 300 109 260 1.484375 60 126 0.796875 60 900 132 338 37.484375 87 180 24.359375 60 2700 176 434 882.109375 116 238 581.953125 61 300 104 246 1.53125 59 124 0.796875 61 900 96 225 26.609375 85 176 24.1875 61 2700 66 154 324.140625 105 216 521.9375 62 300 104 259 1.390625 61 146 0.8125 62 900 89 232 25.265625 99 220 27.625 62 2700 126 314 623.578125 105 226 521.1875 63 300 143 362 2.046875 147 308 2.046875 63 900 97 259 27.59375 173 365 49.28125 63 2700 186 465 933.5 199 422 1005.09375 64 300 40 88 0.53125 45 98 0.578125 64 900 28 62 7.078125 32 70 8.296875 64 2700 30 66 135.65625 33 72 150.015625 65 300 22 48 0.359375 22 48 0.4375 65 900 19 45 5.046875 19 45 5.1875 65 2700 18 40 79.125 18 40 80.765625 66 300 611 1233 11.171875 618 1238 11.6875 66 900 1000 2003 284.8125 1000 2002 286.65625 66 2700 1000 2003 4583.015625 1000 2002 4587.296875 67 300 6 21 0 6 21 0 67 900 12 33 0.25 12 33 0.21875 67 2700 10 29 1.203125 13 59 1.421875 68 300 24 50 0.234375 29 60 0.328125 68 900 25 52 6.453125 31 64 8.375 68 2700 27 56 126.4375 33 68 156.328125
 MTPSBFGS-YWL SBFGS-WWP No. Dim NI NFG CPU time NI NFG CPU time 69 300 26 56 0.265625 25 54 0.21875 69 900 28 60 7.546875 25 54 6.734375 69 2700 29 62 136.421875 25 54 119.25 70 300 31 84 0.5 30 90 0.4375 70 900 40 103 11 32 88 8.46875 70 2700 33 91 146.765625 53 122 245.71875 71 300 342 894 4.171875 190 381 2.203125 71 900 1000 2572 617.34375 534 1069 145.265625 71 2700 1000 2591 7501.1875 1000 2001 4617.703125 72 300 124 313 2.53125 109 284 2.265625 72 900 283 762 94.0625 311 781 107.890625 72 2700 843 2128 4629.25 871 2163 4782.0625 73 300 95 192 1.0625 95 192 1.125 73 900 168 338 46.984375 169 340 46.765625 73 2700 296 594 1477.046875 292 586 1462.71875 74 300 37 76 0.375 36 74 0.34375 74 900 50 102 13.40625 50 102 13.484375 74 2700 81 164 396.390625 81 164 399.5625
4.2. Muskingum Model in Engineering Problems

In this subsection, we present the Muskingum model, and it has the following form:

Muskingum model [50]:whose symbolic representation is as follows: is the storage time constant, is the weight coefficient, is an extra parameter, is the observed inflow discharge, is the observed outflow discharge, is the total time, and is the time step at time .

The observed data of the experiment are obtained from the process of flood runoff from Chenggouwan and Linqing of Nanyunhe in the Haihe Basin, Tianjin, China. Select the initial point and the time step . The concrete values of and for the years 1960, 1961, and 1964 are listed in [51]. The test results are presented in Table 7.

 Algorithm BFGS [52] 10.8156 0.9826 1.0219 HIWO [50] 13.2813 0.8001 0.9933 MTPSBFGS 11.1849 1.0000 0.9996

Figures 46 and Table 7 imply the following three conclusions: (i) based on the Muskingum model, the efficiency of the MTPSBFGS method is wonderful, and numerical performance of these three algorithms is fantastic. (ii) Compared to other similar methods, the final points (, , and ) of the MTPSBFGS method are competitive. (iii) Due to the endpoints of these three methods being different, the Muskingum model may have more approximation optimum points.

5. Conclusion

A modified two parameter scaled BFGS method and the Yuan-Wei-Lu line search technology are introduced in this paper. By scaling the first two terms and the third term of the standard BFGS method with different positive parameters, a new two parameter scaled BFGS method is proposed. In this method, the new value of is given to guarantee better properties of the new scaled BFGS method. With Yuan-Wei-Lu line search, the proposed BFGS method is globally convergent. Numerical results indicate that the modified two parameter scaled BFGS method outperforms the standard BFGS method and even the same type of the BFGS method. As for the longer-term work, there are several points to consider: (1) are there some new values of , , and that make the BFGS method based on the update formula (17) perform better? (2) Whether the new scaled method combined with other line search have also great theoretical results. (3) Some new engineering problems based on the BFGS-type method are worth studying.

Data Availability

The data used to support this study are included within this article.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant no. 11661009), the High Level Innovation Teams and Excellent Scholars Program in Guangxi Institutions of Higher Education (Grant no. (2019)52), the Guangxi Natural Science Key Fund (Grant no. 2017GXNSFDA198046), and the Guangxi Natural Science Foundation (Grant no. 2020GXNSFAA159069).

References

1. R. H. Byrd, S. L. Hansen, J. Nocedal, and Y. Singer, “A stochastic quasi-Newton method for large-scale optimization,” SIAM Journal on Optimization, vol. 26, no. 2, pp. 1008–1031, 2016. View at: Publisher Site | Google Scholar
2. A. S. Lewis and M. L. Overton, “Nonsmooth optimization via quasi-Newton methods,” Mathematical Programming, vol. 141, no. 1-2, pp. 135–163, 2013. View at: Publisher Site | Google Scholar
3. M. S. Salim and A. I. Ahmed, “A family of Quasi-Newton methods for unconstrained optimization problems,” Optimization, vol. 67, no. 10, pp. 1717–1727, 2018. View at: Publisher Site | Google Scholar
4. Z. Wei, G. Li, and L. Qi, “New quasi-Newton methods for unconstrained optimization problems,” Applied Mathematics and Computation, vol. 175, no. 2, pp. 1156–1188, 2006. View at: Publisher Site | Google Scholar
5. Z. Wei, G. Yu, G. Yuan, and Z. Lian, “The superlinear convergence of a modified BFGS-type method for unconstrained optimization,” Computational Optimization and Applications, vol. 29, no. 3, pp. 315–332, 2004. View at: Publisher Site | Google Scholar
6. G. Yuan, Z. Sheng, B. Wang, W. Hu, and C. Li, “The global convergence of a modified BFGS method for nonconvex functions,” Journal of Computational and Applied Mathematics, vol. 327, pp. 274–294, 2018. View at: Publisher Site | Google Scholar
7. W. Zhou and L. Zhang, “Global convergence of the nonmonotone MBFGS method for nonconvex unconstrained minimization,” Journal of Computational and Applied Mathematics, vol. 223, no. 1, pp. 40–47, 2009. View at: Publisher Site | Google Scholar
8. W. Zhou and X. Chen, “Global convergence of a new hybrid Gauss-Newton structured BFGS method for nonlinear least squares problems,” SIAM Journal on Optimization, vol. 20, no. 5, pp. 2422–2441, 2010. View at: Publisher Site | Google Scholar
9. D.-H. Li and M. Fukushima, “A modified BFGS method and its global convergence in nonconvex minimization,” Journal of Computational and Applied Mathematics, vol. 129, no. 1-2, pp. 15–35, 2001. View at: Publisher Site | Google Scholar
10. D.-H. Li and M. Fukushima, “On the global convergence of the BFGS method for nonconvex unconstrained optimization problems,” SIAM Journal on Optimization, vol. 11, no. 4, pp. 1054–1064, 2001. View at: Publisher Site | Google Scholar
11. L. Liu, Z. Wei, and X. Wu, “The convergence of a new modified BFGS method without line searches for unconstrained optimization or complexity systems,” Journal of Systems Science and Complexity, vol. 23, no. 4, pp. 861–872, 2010. View at: Publisher Site | Google Scholar
12. Y. Xiao, Z. Wei, and Z. Wang, “A limited memory BFGS-type method for large-scale unconstrained optimization,” Computers & Mathematics with Applications, vol. 56, no. 4, pp. 1001–1009, 2008. View at: Publisher Site | Google Scholar
13. C. Zhu, R. H. Byrd, P. Lu, and J. Nocedal, “Algorithm 778: l-BFGS-B: fortran subroutines for large-scale bound-constrained optimization,” ACM Transactions on Mathematical Software, vol. 23, no. 4, pp. 550–560, 1997. View at: Publisher Site | Google Scholar
14. W. Zhou, “A modified BFGS type quasi-Newton method with line search for symmetric nonlinear equations problems,” Journal of Computational and Applied Mathematics, vol. 367, Article ID 112454, 2020. View at: Publisher Site | Google Scholar
15. L. Zhang and H. Tang, “A hybrid MBFGS and CBFGS method for nonconvex minimization with a global complexity bound,” Pacific Journal of Optimization, vol. 14, no. 4, pp. 693–702, 2018. View at: Google Scholar
16. W. Zhou and L. Zhang, “A modified Broyden-like quasi-Newton method for nonlinear equations,” Journal of Computational and Applied Mathematics, vol. 372, Article ID 112744, 2020. View at: Publisher Site | Google Scholar
17. M. J. D. Powell, “Some global convergence properties of a variable metric algorithm for minimization without exact line searches,” SIAM-AMS Proceedings, vol. 9, pp. 53–72, 1976. View at: Google Scholar
18. R. H. Byrd, J. Nocedal, and Y.-X. Yuan, “Global convergence of a cass of quasi-Newton methods on convex problems,” SIAM Journal on Numerical Analysis, vol. 24, no. 5, pp. 1171–1190, 1987. View at: Publisher Site | Google Scholar
19. L. C. W. Dixon, “Variable metric algorithms: necessary and sufficient conditions for identical behavior of nonquadratic functions,” Journal of Optimization Theory and Applications, vol. 10, no. 1, pp. 34–40, 1972. View at: Publisher Site | Google Scholar
20. A. Griewank, “The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients,” Mathematical Programming, vol. 50, no. 1–3, pp. 141–175, 1991. View at: Publisher Site | Google Scholar
21. M. J. D. Powell, “On the convergence of the variable metric algorithm,” IMA Journal of Applied Mathematics, vol. 7, no. 1, pp. 21–36, 1971. View at: Publisher Site | Google Scholar
22. W. F. Mascarenhas, “The BFGS method with exact line searches fails for non-convex objective functions,” Mathematical Programming, vol. 99, no. 1, pp. 49–61, 2004. View at: Publisher Site | Google Scholar
23. Y.-H. Dai, “Convergence properties of the BFGS algorithm,” SIAM Journal on Optimization, vol. 13, no. 3, pp. 693–701, 2006. View at: Publisher Site | Google Scholar
24. G. Yuan and Z. Wei, “Convergence analysis of a modified BFGS method on convex minimizations,” Computational Optimization and Applications, vol. 47, no. 2, pp. 237–255, 2010. View at: Publisher Site | Google Scholar
25. J. Nocedal, “Theory of algorithms for unconstrained optimization,” Acta Numerica, vol. 1, pp. 199–242, 1992. View at: Publisher Site | Google Scholar
26. S. S. Oren and D. G. Luenberger, “Self-scaling variable metric (SSVM) algorithms, part I: criteria and sufficient conditions for scaling a class of algorithms,” Management Science, vol. 20, no. 5, pp. 845–862, 1974. View at: Publisher Site | Google Scholar
27. J. Nocedal and Y.-X. Yuan, “Analysis of self-scaling quasi-Newton method,” Mathematical Programming, vol. 61, no. 1–3, pp. 19–37, 1993. View at: Publisher Site | Google Scholar
28. M. Al-Baali, “Analysis of a family of self-scaling quasi-Newton methods,” Tech. Rep., Department of Mathematics and Computer Science, United Arab Emirates University, Al Ain, UAE, 1993, Technical report. View at: Google Scholar
29. Y.-X. Yuan, “A modified BFGS algorithm for unconstrained optimization,” IMA Journal of Numerical Analysis, vol. 11, no. 3, pp. 325–332, 1991. View at: Publisher Site | Google Scholar
30. M. J. D. Powell, “How bad are the BFGS and DFP methods when the objective function is quadratic?” Mathematical Programming, vol. 34, no. 1, pp. 34–47, 1986. View at: Publisher Site | Google Scholar
31. J. Barzilai and J. M. Borwein, “Two-point step size gradient methods,” IMA Journal of Numerical Analysis, vol. 8, no. 1, pp. 141–148, 1988. View at: Publisher Site | Google Scholar
32. W. Y. Cheng and D. H. Li, “Spectral scaling BFGS method,” Journal of Optimization Theory and Applications, vol. 146, no. 2, pp. 305–319, 2010. View at: Publisher Site | Google Scholar
33. N. Andrei, “An adaptive scaled BFGS method for unconstrained optimization,” Numerical Algorithms, vol. 77, no. 2, pp. 413–432, 2017. View at: Publisher Site | Google Scholar
34. N. Andrei, “A double parameter scaled BFGS method for unconstrained optimization,” Journal of Computational and Applied Mathematics, vol. 332, pp. 26–44, 2018. View at: Publisher Site | Google Scholar
35. Y.-H. Dai and C.-X. Kou, “A nonlinear conjugate gradient algorithm with an optimal property and an improved wolfe line search,” SIAM Journal on Optimization, vol. 23, no. 1, pp. 296–320, 2013. View at: Publisher Site | Google Scholar
36. Z. Dai, X. Dong, J. Kang, and L. Hong, “Forecasting stock market returns: new technical indicators and two-step economic constraint method,” The North American Journal of Economics and Finance, vol. 53, Article ID 101216, 2020. View at: Publisher Site | Google Scholar
37. Z. Dai and H. Zhu, “A modified Hestenes-Stiefel-type derivative-free method for large-scale nonlinear monotone equations,” Mathematics, vol. 8, no. 2, p. 168, 2020. View at: Publisher Site | Google Scholar
38. G. Yuan, X. Wang, and Z. Sheng, “Family weak conjugate gradient algorithms and their convergence analysis for nonconvex functions,” Numerical Algorithms, vol. 84, no. 3, pp. 935–956, 2020. View at: Publisher Site | Google Scholar
39. G. Yuan, J. Lu, and Z. Wang, “The PRP conjugate gradient algorithm with a modified WWP line search and its application in the image restoration problems,” Applied Numerical Mathematics, vol. 152, pp. 1–11, 2020. View at: Publisher Site | Google Scholar
40. G. Yuan, T. Li, and W. Hu, “A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems,” Applied Numerical Mathematics, vol. 147, pp. 129–141, 2020. View at: Publisher Site | Google Scholar
41. G. Yuan, Z. Wei, and Y. Yang, “The global convergence of the Polak-Ribière-Polyak conjugate gradient algorithm under inexact line search for nonconvex functions,” Journal of Computational and Applied Mathematics, vol. 362, pp. 262–275, 2019. View at: Publisher Site | Google Scholar
42. L. Zhang, “A derivative-free conjugate residual method using secant condition for general large-scale nonlinear equations,” Numerical Algorithms, vol. 83, no. 4, pp. 1277–1293, 2020. View at: Publisher Site | Google Scholar
43. W. Zhou, “A short note on the global convergence of the unmodified PRP method,” Optimization Letters, vol. 7, no. 6, pp. 1367–1372, 2013. View at: Publisher Site | Google Scholar
44. W. Sun and Y. Yuan, Optimization Theory and Methods, Springer US, New York, NY, USA, 2006.
45. R. H. Byrd, D. C. Liu, and J. Nocedal, “On the behavior of broyden’s class of quasi-Newton methods,” SIAM Journal on Optimization, vol. 2, no. 4, pp. 533–557, 1992. View at: Publisher Site | Google Scholar
46. M. J. D. Powell, “Updating conjugate directions by the BFGS formula,” Mathematical Programming, vol. 38, no. 1, pp. 29–46, 1987. View at: Publisher Site | Google Scholar
47. I. Bongartz, A. R. Conn, N. Gould, and P. L. Toint, “CUTE: constrained and unconstrained testing environment,” ACM Transactions on Mathematical Software, vol. 21, no. 1, pp. 123–160, 1995. View at: Publisher Site | Google Scholar
48. J. J. Moré, B. S. Garbow, and K. E. Hillstrom, “Testing unconstrained optimization software,” ACM Transactions on Mathematical Software (TOMS), vol. 7, no. 1, pp. 17–41, 1981. View at: Publisher Site | Google Scholar
49. Y. Yuan and W. Sun, Theory and Methods of Optimization, Science Press of China, Beijing, China, 1999.
50. A. Ouyang, L.-B. Liu, Z. Sheng, and F. Wu, “A class of parameter estimation methods for nonlinear Muskingum model using hybrid invasive weed optimization algorithm,” Mathematical Problems in Engineering, vol. 2015, Article ID 573894, 15 pages, 2015. View at: Publisher Site | Google Scholar
51. A. Ouyang, Z. Tang, K. Li, A. Sallam, and E. Sha, “Estimating parameters of Muskingum model using an adaptive hybrid PSO algorithm,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 28, pp. 1–29, 2014. View at: Publisher Site | Google Scholar
52. Z. W. Geem, “Parameter estimation for the nonlinear Muskingum model using the BFGS technique,” Journal of Irrigation and Drainage Engineering, vol. 132, no. 5, pp. 474–478, 2006. View at: Publisher Site | Google Scholar

Copyright © 2020 Pengyuan Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.