Research Article | Open Access
Pengjie Liu, Jinbao Jian, Xianzhen Jiang, "A New Conjugate Gradient Projection Method for Convex Constrained Nonlinear Equations", Complexity, vol. 2020, Article ID 8323865, 14 pages, 2020. https://doi.org/10.1155/2020/8323865
A New Conjugate Gradient Projection Method for Convex Constrained Nonlinear Equations
The conjugate gradient projection method is one of the most effective methods for solving large-scale monotone nonlinear equations with convex constraints. In this paper, a new conjugate parameter is designed to generate the search direction, and an adaptive line search strategy is improved to yield the step size, and then, a new conjugate gradient projection method is proposed for large-scale monotone nonlinear equations with convex constraints. Under mild conditions, the proposed method is proved to be globally convergent. A large number of numerical experiments for the presented method and its comparisons are executed, which indicates that the presented method is very promising. Finally, the proposed method is applied to deal with the recovery of sparse signals.
Solving a system of nonlinear equations can be transformed as an optimization problem, which is widely applied in many fields of sciences and engineering, for instance, the economic equilibrium problem , the neural networks problem , the financial problem [3, 4], the chemical equilibrium system , and the compressed sensing problem [6, 7].
In this paper, the following system of constrained monotone nonlinear equations is considered:where is a nonempty closed convex set of and is a monotone mapping, namely,
Many algorithms have been proposed to deal with (1) during the past few decades (see, e.g., [9–16]), such as the projected Newton method , the projected quasi-Newton method [10–13], the Levenberg–Marquardt method , the trust region method , and the Lagrangian global method . As we know, these methods converge rapidly if the sufficiently good initial points are chosen. However, they are not well-suited for solving large-scale constrained nonlinear equations due to the computation of the Jacobian matrix or its approximation at each iteration. Therefore, in the past few years, the projected derivative-free method (PDFM) has become more and more popular, i.e., the spectral gradient projection method [17–19], the multivariate spectral gradient-type projection method [20, 21], and the conjugate gradient projection method (CGPM) [22–31], and more other PDFM can be seen in references [32, 33].
In this work, we concentrate on studying CGPM for large-scale nonlinear equations with convex constraints. We aim to establish a more efficient CGPM by improving the search direction and the line search rule and use the proposed CGPM to deal with the problem of sparse signals recovery.
The contributions of this article are listed as follows:(i)To guarantee the search direction satisfying the sufficient descent condition and trust region property independent of any line searches, a new conjugate parameter is proposed;(ii)Based on classic line searches for nonlinear equations, an adaptive line search is improved to seek suitable step size easily;(iii)Under general assumptions, the convergence analysis of the proposed algorithm is proved;(iv)The reported numerical experiments show that our method is promising for solving large-scale nonlinear constrained equations and handling the problem of recovering sparse signals.
The remainder of this paper is organized as follows. In Section 2, a new search direction and an adaptive line search are proposed, and the corresponding algorithm is given. The global convergence is studied in Section 3. In Section 4, the numerical experiments for the proposed algorithm and its comparisons are performed, and the corresponding results are reported. Application of the proposed algorithm in compressed sensing is introduced in Section 5. In Section 6, a conclusion for this work is given.
2. A New CGPM Algorithm
The CGPM has been attracting extensive attention, since it not only inherits the advantages of the conjugate gradient method (CGM) with a simple algorithm structure, rapid convergence, and low storage requirements but also uses no any jacobian information of the equation in practice. To the best of our knowledge, the computation cost of CGPM mainly exists in the process of generating the search direction and computing the step size. Therefore, in the following part, we design our search direction by the CGM and give an improved inexact line search to yield the step size.
2.1. The New Search Direction Yielded by CGM
It is well-known that the search direction of the classical CGM is generated bywhich is decided by the conjugate parameter . Usually, a different leads to a different search direction.
In the recent years, many scholars have made efforts to extend the CGM to solve the large-scale nonlinear monotone equations system. For example, based on the hybrid conjugate parameter in , Sun and Liu  proposed a modified conjugate parameter as follows:where and extended it to solve problem (1).
Recently, Tsegay et al.  gave a new CGM with sufficient descent property, that is,
It is interesting that the search direction with have better theoretical properties, that is, it satisfies the sufficient descent condition and trust region property, automatically.
2.2. An Improved Adaptive Line Search Strategy
For an efficient CGPM, choosing an inexpensive line search is a key technique. To this end, many researchers try to exploit an inexact line search strategy to obtain step size with minimal cost. Zhang and Zhou  adopt an Armijo-type line search procedure, that is, the step size , such thatwhere is an initial guess for the step size, , and is a positive constant. Li and Li  obtained the step size by the following line search:which was originally proposed by Solodov and Svaiter . Guo and Wan  proposed an adaptive line search, i.e., the step size satisfied the following inequality:
Obviously, when is far from the solutions of problem (1) and the is too large, it follows from (9) that the step size becomes small, which increases the computation cost. A similar case can appear for the line search (8) when is close to the solution set of problem (1) and is too large. However, it is worth noting that the line search (10) can overcome the previously mentioned weaknesses and take advantage of line searches (8) and (9).
Inspired by , we introduce another new adaptive line search strategy with a disturbance factor, that is, taking , such thatwhere , is an initial guess for the step size, , and . Here, is a disturbance factor, which can adjust the size of the right side of the line search (11) and further reduce the computation cost.
Remark 1. In fact, for a given , is too large if is far away from the solution set, namely, , and then, the new line search (11) is similar to (8) in performance. Otherwise, when is close to the solution set, approaches 0 and so approaches , and then, the new line search (11) comes back to (9).
3. Convergence Property
In order to obtain some important properties and convergence property of Algorithm 1, the following basic assumptions are necessary. Assumption H: (H1) The solution set of system (1), denoted by , is nonempty, and the mapping is monotone on . (H2) The mapping is -Lipschitz continuous on , i.e., there exists a constant such that
The well-known nonexpansive property of the projection operator  is reviewed in the following lemma.
Lemma 1 (see ). Let be a nonempty closed convex set. Then,
Therefore, the projection operator is L-Lipschitz continuous on .
The following lemma shows that the search direction yielded by equation in step 2 in Algorithm 1 satisfies the sufficient descent condition and possesses some important properties.
Lemma 2. Suppose that Assumption H holds, then the search direction generated by equation in step 2 in Algorithm 1 satisfies the sufficient descent condition,and , for some positive constants and .
Proof. For , it is easy to know that and Lemma 2 holds. To proceed, we consider the case . If , it follows from equation in step 2 in Algorithm 1 that . Otherwise, multiplying both sides of equation in step 2 in Algorithm 1 by , from (6) and (7), we havewhich shows that the sufficient descent property (14) holds by taking . Again, according tothe following relation holds:and then, .
On the other hand, it follows from (7) and equation in step 2 in Algorithm 1 thatand the proof is completed.
The next lemma not only indicates that the line search strategy (11) is well-defined but also provides a lower bound for step size .
Lemma 3. (i)Let the sequences and be generated by Algorithm 1; then, there exists a step size satisfying the line search (11)(ii)Suppose that Assumption H holds; then, the step size yielded by Algorithm 1 satisfies
Proof. (i)Suppose that for any nonnegative integer , (11) does not hold at the -th iterate,then From the continuity of and , let , and it is clear that which contradicts (14). The proof is completed.(ii)For the second part, it is clear that if , then (19) holds. If , is computed by the backtracking process in Algorithm 1. Let , and then, does not satisfy (11), namely,where . It follows from (12), (14), and (22) thatThen,which completes the proof.
The following lemma is necessary to analyze the global convergence of Algorithm 1.
Lemma 4. Suppose that Assumptions H holds, let and be generated by Algorithm 1, and let be any given solution for system (1), i.e., . Then, the sequence is convergent, and sequences and are both bounded. Furthermore, it holds that
Proof. In view of the definition of and (11), we know thatOn the other hand, taking Assumption (H1) and into consideration, we haveAccording to equation in step 4 in Algorithm 1, Assumption (H1), Lemma 1, and (26) and (27), it follows thatwhich shows that the inequalities hold, that is, the sequence is monotone nonincreasing and bounded below. Hence, is convergent. Furthermore, the boundedness of is obtained.
By Lemma 2, it holds that is bounded and so is . Without the loss of generality, there exists a constant such that . If , then ; otherwise, . Hence, the following relation holds:This together with (28) implies thatThus, this further implies , and the proof is completed.
Next, based on Assumption H and Lemmas 1–4, the global convergence of the proposed algorithm is established.
Theorem 1. Suppose that Assumption H holds and the sequences be generated by the Algorithm 1; then,
Furthermore, the whole sequence converges to a solution of system (1).
Proof. First, by contradiction, suppose that relation (31) is not true; then, there exists a constant such thatAgain, the following inequality comes directly from Lemma 2:This together with (25) shows thatIn addition, from Lemmas 2 and 3 (ii) and the boundedness of , the following relation holds:which contradicts (34). Therefore, (31) is true.
Second, (31) shows that there exists an infinite index set such that . Again, the is bounded and is a closed set, so without the loss of generality, suppose that . It follows from the continuity of thatwhich shows that .
Finally, noticing that from Lemma 4, it follows that the sequence is convergent, namely,which implies that the whole sequence converges to , and the proof is completed.
4. Numerical Experiments
In this section, the numerical performances of Algorithm 1 (LJJ CGPM) for solving convex constrained nonlinear equations are tested and reported by the following two subsections.
4.1. Experimental Setup
In order to illustrate the effectiveness of the LJJ CGPM, we compare it with two recent CGPMs. Specifically, eight large-scale examples are solved by the LJJ CGPM method, PDY method , and ATTCGP method  in the same calculating environment. All codes were written in Matlab R2014a and run on a DELL with 4 GB RAM memory and Windows 10 operating system.
For the LJJ CGPM, we use (6) and (11) as the conjugate parameter of search direction and the line search rule, respectively, and the parameters in the LJJ CGPM are chosen as . For the PDY  and the ATTCGP  methods, the search direction, line search rule, and selection of parameters are consistent with the original literature, respectively.
For all methods, the computation will be terminated when one of the following criteria are satisfied:where “Itr” refers to the total number of iterations. Defineand the tested functions are listed as follows.
Problem 2. (see Yu et al. ). Setfor and .
Problem 4. (see Zhou and Li ). Set , for and .
Problem 5. (see Gao and He ). Setfor and .
Problem 6. (see Ou and Li ). Setfor and .
Problem 7. (see Gao and He ). Set , for and .
Problem 8. (see Gao and He ). Setfor and .
The new iterate points yielded by the quadratic program solver quadprog. m are taken from the Matlab optimization toolbox. Problems 1–8 are tested with seven initial points . Here, the dimension of problems is chosen as , and , respectively.
The comparison of data is listed in Tables 1–8, where “Init” means the initial point, “n” is the dimension of the problem, “NF” denotes the number of function evaluations, “Tcpu” denotes the CPU time, and “” is the final value of when the program is stopped.
In addition, in order to show the numerical performance clearly, we adopt the profiles introduced by Dolan and Morè  to compare the performance on Tcpu, NF, and Itr, respectively. A brief explanation of the performance figures is as follows. Denote the whole set of test problems by and the set of solvers by . Let be the Tcpu (or the or the ) required to solve problem by solver , and the comparison results between different solvers are based on the performance ratio defined fromand the performance profile for each solver is defined bywhere size A means the number of elements in the set A. Then, is the probability for solver that a performance ratio is within a factor . is the (cumulative) distribution function for the performance ratio. Clearly, the top curved shape of the method is a winner. For details about the performance profile, see .