A New Hybrid Conjugate Gradient Projection Method for Solving Nonlinear Monotone Equations

Fang, Minglei; Wang, Min; Ding, Defeng

doi:https://doi.org/10.1155/2022/5540504

Journal of Mathematics

On this page

Abstract Introduction Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2022 | Article ID 5540504 | https://doi.org/10.1155/2022/5540504

A New Hybrid Conjugate Gradient Projection Method for Solving Nonlinear Monotone Equations

Minglei Fang,¹Min Wang,¹and Defeng Ding¹

Academic Editor: Predrag S. Stanimirović

Received04 Aug 2022

Revised30 Sept 2022

Accepted26 Oct 2022

Published14 Nov 2022

Abstract

In this study, we propose a new modified hybrid conjugate gradient projection method with a new scale parameter for solving large-scale nonlinear monotone equations. The proposed method includes two major features: projection techniques and sufficient descent property independent of line search technique. Global convergence of the proposed method is proved under some suitable assumptions. Finally, numerical results illustrating the robustness of the suggested strategy and its comparisons are shown.

1. Introduction

The following system of constrained nonlinear monotone equations is considered:where is a nonempty closed convex set and is continuous and monotone function. The monotone property of means that

In many contemporary domains, some problems can be turned into nonlinear monotonic equation problems, such as variational inequality problems [1], image restoration problems [2], signal reconstruction problems [3], financial forecasting problems [4, 5], and optimal power flow control problems in power [6]. It is very necessary to solve nonlinear monotone equation.

At present, there are many ways to solve problem (1), such as the Newtons method [7], Levenberg–Marquardt method [8], and various variants of the method. Although iterative approaches are recognized for their simplicity and good convergence, each iteration of these methods requires a substantial amount of space to compute and store the Jacobian or comparable to the Jacobian, which is not conducive to solving large-scale nonlinear monotone equations. In order to efficiently solve problem (1) and avoid solving a linear system of equations at each iteration, the emergence of the derivative-free method is necessary [9–11].

In recent years, a number of researchers have proposed the conjugate gradient method in conjunction with the projection method [12] for solving large-scale monotone nonlinear equations. Sabi’u et al. [13] proposed the Hager–Zhang conjugate gradient method for nonlinear monotone equations using singular value analysis and proposed two adaptive parameter choices: the first by minimizing the Frobenius condition number of the search direction matrix; the other is achieved by minimizing the difference between the maximum and singular values, combined with the projection technique, using some appropriate assumptions to demonstrate that the method satisfies the global convergence, see [14, 15]. Yin et al. [16] proposed a hybrid three-term conjugate gradient projection method, whose search direction is close to the search direction generated by the memoryless BFGS method and possesses a descending characteristic independent of the line search technique as well as a trust region characteristic. Using the adaptive line search technique, the global convergence of the method is established under certain mild conditions, and numerical experiments demonstrate that the method inherits the beneficial properties of the three-term conjugate gradient method and the hybrid conjugate gradient method [17, 18]. Zhou et al. [19] proposed a novel hybrid PRPFR conjugate gradient method with sufficient descent and trust region properties. The method can be considered as a convex combination of the PRP method and the FR method, while using the hyperplane projection technique. In accordance with the acceleration step size, global convergence is attained with the help of some suitable assumptions. Experiments with numbers demonstrate that the PRPFR method is more competitive for solving nonlinear equations and image restoration problems [20, 21].

In this paper, we propose a new hybrid conjugate gradient projection method to solve (1). Interestingly, the new search direction has better theoretical properties, that is, it automatically satisfies the sufficient descent condition and the trust region property. The rest of the paper is organized as follows. In Section 2, we detail the motivation for this paper and propose a new algorithm. In Section 3, we demonstrate that the search direction has sufficient descent property and trust region property, and we obtain global convergence under a few moderate conditions. In Section 4, we solve six large-scale nonlinear equations numerically to demonstrate the effectiveness of the proposed method. In Section 5, we draw the conclusion.

2. Motivation and Algorithm

This section quickly reviews the conjugate gradient method’s general formula, followed by a new hybrid conjugate gradient approach based on adaptive line search. Finally, we combine the suggested method with the projection technique to solve unconstrained optimization problems.

Due to its ease of use and storage, the conjugate gradient approach has been effectively applied to the following unconstrained optimization problems:where is a continuously differentiable function.

Generally, these methods generate a sequence of iterates recurrently bywhere (it is the step size calculated by performing some suitably precise or imprecise line search) and is the search direction defined by

The term is a scalar known as the conjugate gradient parameter. Different choices for the conjugate gradient update lead to different conjugate gradient methods. There are some famous conjugate gradient methods such as the FR method [22], the PRP method [23], the DY method [24], the LS method [25], the HS method [26], the CD method [27], and so on. The parameters we mentioned are as follows:where and the symbol stands for the Euclidean norm.

An important class of conjugate gradient methods, hybrid conjugate gradient methods, has been proposed by many scholars, in order to obtain a conjugate gradient method with better performance than the classical one. For example, Djordjević [28] combined the LS method and FR method by convex combination form to get conjugate gradient parameters:using the good practical performance of the LS method and the strong convergence of the FR method.

Soon after, Zhou et al. [19] gave a variant of the PRP method and FR method:where . Similarly, define a new parameter

In [29], based on the MMWU method [30] and RMAR method [31], Fanar and Ghada proposed a new hybrid conjugate gradient method (HFG) as follows:

This technique is a convex mixture of two types of conjugate gradient algorithms, and the search direction very easily satisfied sufficient descent, besides conforming to Newton direction under appropriate conditions. The global convergence of the proposed method can be established when a strong Wolfe line search is performed. Shockingly, this hybrid method not only performs better than the classical conjugate gradient method but also outperforms some complex conjugate gradient methods in many problems.

Inspired by above, a new convex combination is proposed as follows:where , and the choice of the parameter satisfies the conjugate condition in each iteration: .

Clearly,

Multiplying from the right both sides of the transposed equation by , we getand with some mathematical calculation, we obtain

If is outside the interval [0, 1], to maintain the convex combination in (11), it can be fixed by

For line search, the key is to obtain the step size at the lowest cost. Therefore, the most widely used line search proposed by Solodov and Svaiter [12] was used, for which the step size is computed as such that

Another commonly used technique is the line search technique proposed by Zhang and Zhou [32], and the step size satisfied the following inequality:

Recently, an adaptive line search approach that takes into consideration a disturbance component was presented by Liu et al. [33] and Guo and Wan [34], i.e., the step size satisfied the following inequality:where and . If , then ; otherwise, if , then .

To establish the algorithm, we introduce the definition of projection operator. Let be a nonempty closed convex set of ; then,

Also, it satisfies the nonexpansive property

We now go into great depth about our algorithm, based on the abovementioned preliminaries.

3. Convergence Analysis

In order to show the global convergence of Algorithm 1, the following assumptions need to be established: (H 3.1) The solution set of problem (1) is nonempty. (H 3.2) The mapping is Lipschitz continuous on , i.e., there exists a constant such that

	Step 0. Select an initial point and the parameters , ,
	. Let .
	Step 1. Compute , if , stop. Otherwise, go to Step 2.
	Step 2. Compute by (5)–(12), and choose that satisfies inequality (19).
	Step 3. Set . If , stop and let . Otherwise, compute the next iterate by
	where
	Step 4. Set , and go to Step 1.

The following lemma demonstrates that the direction of the search satisfies sufficient descent condition and possesses trust region property independent on any line search.

Lemma 1. For all , if the search direction is generated by Algorithm 1, then the following two properties can be satisfied:

Proof. For , it is easy to know that and (23) and (24) hold.
For , it can be broken down into three cases for discussion:(i)Firstly, if , then . If , then , and if , then . It follows that . Multiplying both sides of (5) by , from (11) and (12), we get According to (5), it is easy to obtain that(ii)Secondly, if , then . As in the case of the first case, we get(iii)Finally, for there exist , in which , and we can simply rewrite formula (5): Then, we can get the relationship among , and : From (26) and (28), we have Obviously, the descending property can be obtained by combining the step relationship (30) with (25) and (27), and we getThe proof is completed.
The following lemma shows that Algorithm 1 can get a step size and stop within finite steps, which shows that this line search is reasonable.

Lemma 2. If assumptions (H3.1) and (H3.2) hold, then Algorithm 1 stops with a finite number of iterations.

Proof. Suppose that the assumptions are invalid or that Algorithm 1 is not terminated. In this case, there exists a constant such thatSuppose there exists , so that line search (19) does not hold. Then, for all , let , and we havewhere . It can be obtained by assumption (H3.2) and formula (23) thatUsing (24), we getThis is contradictory to the definition of . Thus, line search (19) can attain a positive step size in finite steps and we complete the proof.
The following outcomes are required in order to establish that Algorithm 1 has achieved global convergence.

Lemma 3. If assumptions (H3.1) and (H3.2) hold, let and be the sequences generated by Algorithm 1, and is the solution of problem (1); for all , the sequence converges. Then, we have

Proof. First, according to the definition of and line search, we haveFrom (2) and , the following relation holds:Then, combining the relation above with equations in Algorithm 1 and (21), we come to the conclusion that(40) shows the relation , which means that the sequence is decreasing and bounded. Thus, the sequence is convergent, and the sequence is bounded. Combined with assumption (H3.2), we haveTherefore, has an upper bound. Through Lemma 1 and the continuity of , it is obtained that is bounded; furthermore, sequence is bounded. This shows that there is a constant such thatLet , where .
This together with (40) impliesSince , the above formula implies that .
On the basis of the lemma above, we show that the algorithm is globally convergent under certain conditions.

Theorem 1. If assumptions (H3.1) and (H3.2) hold, let sequence and be generated by Algorithm 1. Then, we have

Proof. It can be proved by contradiction. Assuming that formula (44) does not hold, then there is a constant such that . According to , where , and formula (23), we getMoreover,Combining the above formula with Lemma 3, it is easy to show thatWe know that the sequences and are bounded by Lemma 3 and formula (24), so there is a cluster point and an infinite index set such that , and there is a clustering point and an infinite index set such that . There exists a in line search (19) such thatTaking the limit of both sides of the above equation, for all , it turns out thatTaking the limit of both sides of (19), we getObviously, the above two formulas are contradictory. Therefore, the assumption does not hold, that is, .

4. Numerical Results

In this section, in order to ensure the effectiveness of the proposed algorithm (Algorithm 1), specific numerical experiments are given below. We denote the proposed Algorithm 1 by WF, and compare with some existing algorithms. The conjugate gradient method in [35, 36, 37, 38] are denoted by JKL, EMDY, PDY, and HG, respectively. These are derivative-free methods, where JKL and EMDY methods are the state-of-the-art methods. Among them, the JKL method, PDY method, and HG method use the line search of formula (17), and the EMDY method uses the line search of formula (18). All codes used were written in Matlab R2015a and run on PC with 4 GB of RAM and Windows 10 operating system.

For different methods, we choose the optimal parameters: WF: . JKL: . EMDY: . PDY: . HG: .

We choose five methods whose termination condition is or .

The test questions are as follows: Problem 1 [11]: set , for and . Problem 2 [34]: set nonsmooth function , for and . Problem 3 [11]: set logarithmic function for and . Problem 4 [33]: set Problem 5 [33]: set where and . Problem 6 [33]: set , for and .

Problems 1–6 use six initial points: . In addition, choose , , and 30000 as the dimension of the problem.

In addition, we compare the efficacy of the five different approaches by utilizing the performance characteristics that Dolan and Moré [39] have provided. Assuming we have solvers and test problems, compare the performance of solving on the problem with the best performance of any solver on the problem. This method is used to compare and measure how well solver set works on test set . The comparison between different solvers is based on

In order to obtain the performance curve of each solver , is defined as the probability of solver , that is, the performance ratio of is within the factor of the best possible ratio. Its performance is superior to that of the following distribution functions:where size is the number of items in set and . In the event that the solver is unable to find a solution to the problem , we will adjust the ratio to a value that is sufficiently high.

Tables 1–6 display all numerical findings derived from the five approaches. In the tables, “Dim” represents the dimension of the problem, “Initial” represents the initial point, “—” indicates that the method failed to converge inside the iteration termination condition, “NI” refers to the number of iterations, “NF” refers to the number of function evaluation, “CPU” denotes the CPU time, and “” refers to the final value of when the program is stopped. In problem 3, the MEDY method outperforms the WF method at some initial points; however, in problem 6, the MEDY method cannot find the corresponding solution for initial point . In problem 4, the PDY method is superior to the WF method in the number of iterations. It can also be observed that in most of the remaining problems, the FW method has better performance than other methods in terms of the number of iterations, the number of function evaluation and CPU time.

As shown in Figures 1, compared with JKL, EMDY, HG, and PDY methods, the proposed method achieves about 70%, 55%, and 43% wins in terms of the number of iterations, function evaluation, and CPU time, respectively. These demonstrate that when compared to the JKL and MEDY methods, the FW method is more robust in terms of the number of iterations, the number of function evaluation, and CPU time.

(a)

(b)

(c)

5. Conclusion

In this paper, we suggested a way to solve nonlinear monotone equations without using derivatives. This method was a combination of the hybrid conjugate gradient method and the hyperplane projection method. An adaptive technique was used to get the search direction . We showed that the proposed method had global convergence under appropriate conditions. This technique worked well for solving large-scale monotone equations since it required a very small amount of memory. Preliminary numerical findings demonstrated the viability of our strategy. In the future, more experiments will be conducted to prove the performance of the proposed algorithm and we will try to extend the algorithm to practical large-scale nonlinear problems.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This study was supported by the Key Program of University Natural Science Research Fund of Anhui Province (no. KJ2021A0451) and Anhui Provincial Natural Science Foundation (no. 2008085MA01).

References

Y. B. Zhao and D. Li, “Monotonicity of fixed point and normal mappings associated with variational inequality and its application,” SIAM Journal on Optimization, vol. 11, no. 4, pp. 962–973, 2001.
View at: Google Scholar
B. Zhang, Z. Zhu, and S. Li, “A modified spectral conjugate gradient projection algorithm for total variation image restoration,” Applied Mathematics Letters, vol. 27, pp. 26–35, 2014.
View at: Google Scholar
B. Gu, V. S. Sheng, K. Y. Tay, W. Romano, and S. Li, “Incremental support vector learning for ordinal regression,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 7, pp. 1403–1416, 2015.
View at: Publisher Site | Google Scholar
Z. Dai, H. Zhu, and X. Zhang, “Dynamic spillover effects and portfolio strategies between crude oil, gold and Chinese stock markets related to new energy vehicle,” Energy Economics, vol. 109, Article ID 105959, 2022.
View at: Publisher Site | Google Scholar
Z. Dai, T. Li, and M. Yang, “Forecasting stock return volatility: the role of shrinkage approaches in a data-rich environment,” Journal of Forecasting, vol. 41, no. 5, pp. 980–996, 2022.
View at: Publisher Site | Google Scholar
A. J. Wood, B. F. Wollenberg, and G. B. Shebl, Power Generation, Operation, and Control, John Wiley & Sons, Hoboken, NJ, USA, 2013.
M. Al-Baali, E. Spedicato, and F. Maggioni, “Broyden’s quasi-Newton methods for a nonlinear system of equations and unconstrained optimization: a review and open problems,” Optimization Methods and Software, vol. 29, no. 5, pp. 937–954, 2014.
View at: Publisher Site | Google Scholar
J. Fan, “On the Levenberg-Marquardt methods for convex constrained nonlinear equations,” Journal of Industrial and Management Optimization, vol. 9, no. 1, pp. 227–241, 2013.
View at: Publisher Site | Google Scholar
J. Sabi’u, K. Muangchoo, A. Shah, A. B. Abubakar, and L. O. Jolaoso, “A modified PRP-CG type derivative-free algorithm with optimal choices for solving large-scale nonlinear symmetric equations,” Symmetry, vol. 13, no. 2, p. 234, 2021.
View at: Publisher Site | Google Scholar
A. B. Abubakar, J. Sabiu, P. Kumam, and A. Shah, “Solving nonlinear monotone operator equations via modified sr1 update,” Journal of Applied Mathematics and Computing, vol. 67, no. 1-2, pp. 343–373, 2021.
View at: Publisher Site | Google Scholar
H. Feng and T. Li, “An accelerated conjugate gradient algorithm for solving nonlinear monotone equations and image restoration problems,” Mathematical Problems in Engineering, vol. 2020, Article ID 7945467, 12 pages, 2020.
View at: Publisher Site | Google Scholar
M. V. Solodov and B. F. Svaiter, “A globally convergent inexact Newton method for systems of monotone equations,” Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods, Springer, Boston, MA, USA, pp. 355–369, 1998.
View at: Google Scholar
J. Sabi’u, A. Shah, and M. Y. Waziri, “Two optimal Hager-Zhang conjugate gradient methods for solving monotone nonlinear equations,” Applied Numerical Mathematics, vol. 153, pp. 217–233, 2020.
View at: Publisher Site | Google Scholar
J. Sabi’u, A. Shah, M. Y. Waziri, and K. Ahmed, “Modified Hager-Zhang conjugate gradient methods via singular value analysis for solving monotone nonlinear equations with convex constraint,” International Journal of Computational Methods, vol. 18, no. 4, Article ID 2050043, 2021.
View at: Publisher Site | Google Scholar
J. Sabi’u, A. Shah, and M. Y. Waziri, “A modified Hager-Zhang conjugate gradient method with optimal choices for solving monotone nonlinear equations,” International Journal of Computer Mathematics, vol. 99, no. 2, pp. 332–354, 2022.
View at: Publisher Site | Google Scholar
J. Yin, J. Jian, X. Jiang, M. Liu, and L. Wang, “A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications,” Numerical Algorithms, vol. 88, no. 1, pp. 389–418, 2021.
View at: Publisher Site | Google Scholar
J. Sabi’u, K. O. Aremu, A. Althobaiti, and A. Shah, “Scaled three-term conjugate gradient methods for solving monotone equations with application,” Symmetry, vol. 14, no. 5, p. 936, 2022.
View at: Publisher Site | Google Scholar
J. Sabi’u and A. Shah, “An efficient three-term conjugate gradient-type algorithm for monotone nonlinear equations,” RAIRO-Operations Research, vol. 55, pp. S1113–S1127, 2021.
View at: Publisher Site | Google Scholar
Y. Zhou, Y. Wu, and X. Li, “A new hybrid prpfr conjugate gradient method for solving nonlinear monotone equations and image restoration problems,” Mathematical Problems in Engineering, vol. 2020, Article ID 6391321, 13 pages, 2020.
View at: Publisher Site | Google Scholar
G. L. Yuan, Y. J. Zhou, and M. X. Zhang, “A hybrid conjugate gradient algorithm for nonconvex functions and its applications in image restoration problems,” Journal of the Operations Research Society of China, pp. 1–23, 2022.
View at: Publisher Site | Google Scholar
J. Sabi’u, K. Muangchoo, A. Shah, A. B. Abubakar, and K. O. Aremu, “An inexact optimal hybrid conjugate gradient method for solving symmetric nonlinear equations,” Symmetry, vol. 13, no. 10, p. 1829, 2021.
View at: Publisher Site | Google Scholar
R. Fletcher and C. M. Reeves, “Function minimization by conjugate gradients,” The Computer Journal, vol. 7, no. 2, pp. 149–154, 1964.
View at: Publisher Site | Google Scholar
E. Polak and G. Ribiere, “Note sur la convergence de méthodes de directions conjuguées,” Revue Française d’Informatique et de Recherche Opérationnelle, vol. 3, no. 16, pp. 35–43, 1969.
View at: Publisher Site | Google Scholar
Y. H. Dai and Y. Yuan, “A nonlinear conjugate gradient method with a strong global convergence property,” SIAM Journal on Optimization, vol. 10, no. 1, pp. 177–182, 1999.
View at: Publisher Site | Google Scholar
Y. Liu and C. Storey, “Efficient generalized conjugate gradient algorithms, part 1: theory,” Journal of Optimization Theory and Applications, vol. 69, no. 1, pp. 129–137, 1991.
View at: Publisher Site | Google Scholar
M. R. Hestenes and E. Steifel, “Method of conjugate gradients for solving linear equations,” Journal of Research of the National Bureau of Standards, vol. 49, pp. 409–436, 1952.
View at: Google Scholar
R. Fletcher, Practical Methods of Optimization Vol 1: Unconstrained Optimization, Wiley and Sons, New York, NY, USA, 1987.
S. S. Djordjević, “New hybrid conjugate gradient method as a convex combination of LS and FR methods,” Acta Mathematica Scientia, vol. 39, no. 1, pp. 214–228, 2019.
View at: Publisher Site | Google Scholar
F. N. Al-Namat and G. M. Al-Naemi, “Global convergence property with inexact line search for a new hybrid conjugate gradient method,” Open Access Library Journal, vol. 7, Article ID e6048, 2020.
View at: Google Scholar
A. V. Mandara, M. Mamat, M. Y. Waziri, M. A. Mohamed, and U. A. Yakubu, “A new conjugate gradient coefficient with exact line search for unconstrained optimization,” Far East Journal of Mathematical Sciences, vol. 105, no. 2, pp. 193–206, 2018.
View at: Publisher Site | Google Scholar
J. K. Liu and S. J. Li, “New hybrid conjugate gradient method for unconstrained optimization,” Applied Mathematics and Computation, vol. 245, pp. 36–43, 2014.
View at: Publisher Site | Google Scholar
L. Zhang and W. Zhou, “Spectral gradient projection method for solving nonlinear monotone equations,” Journal of Computational and Applied Mathematics, vol. 196, no. 2, pp. 478–484, 2006.
View at: Publisher Site | Google Scholar
P. Liu, J. Jian, and X. Jiang, “A new conjugate gradient projection method for convex constrained nonlinear equations,” Complexity, vol. 2020, Article ID 8323865, 14 pages, 2020.
View at: Publisher Site | Google Scholar
J. Guo and Z. Wan, “A modified spectral PRP conjugate gradient projection method for solving large-scale monotone equations and its application in compressed sensing,” Mathematical Problems in Engineering, vol. 2019, Article ID 5261830, 17 pages, 2019.
View at: Publisher Site | Google Scholar
J. K. Liu, Z. L. Lu, J. L. Xu, S. Wu, and Z. Tu, “An efficient projection-based algorithm without Lipschitz continuity for large-scale nonlinear pseudo-monotone equations,” Journal of Computational and Applied Mathematics, vol. 403, Article ID 113822, 2022.
View at: Publisher Site | Google Scholar
A. Kambheera, A. H. Ibrahim, A. B. Muhammad, A. B. Abubakar, and B. A. Hassan, “Modified Dai-Yuan conjugate gradient method with sufficient descent property for nonlinear equations,” Thai Journal of Mathematics, pp. 145–167, 2022.
View at: Google Scholar
J. Liu and Y. Feng, “A derivative-free iterative method for nonlinear monotone equations with convex constraints,” Numerical Algorithms, vol. 82, no. 1, pp. 245–262, 2019.
View at: Publisher Site | Google Scholar
P. Gao and C. He, “An efficient three-term conjugate gradient method for nonlinear monotone equations with convex constraints,” Calcolo, vol. 55, no. 4, pp. 53–17, 2018.
View at: Publisher Site | Google Scholar
E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming, vol. 91, no. 2, pp. 201–213, 2002.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2022 Minglei Fang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

222

Downloads

330

Citations