Mathematical Problems in Engineering

Volume 2019, Article ID 5976595, 9 pages

https://doi.org/10.1155/2019/5976595

## Modified Three-Term Conjugate Gradient Method and Its Applications

School of Mathematics and Statistics, Qingdao University, Qingdao 266071, China

Correspondence should be addressed to Shouqiang Du; nc.ude.udq@udqs

Received 16 January 2019; Revised 15 March 2019; Accepted 25 March 2019; Published 17 April 2019

Academic Editor: Yann Favennec

Copyright © 2019 Jiankun Liu and Shouqiang Du. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

We propose a modified three-term conjugate gradient method with the Armijo line search for solving unconstrained optimization problems. The proposed method possesses the sufficient descent property. Under mild assumptions, the global convergence property of the proposed method with the Armijo line search is proved. Due to simplicity, low storage, and nice convergence properties, the proposed method is used to solve -tensor systems and a kind of nonsmooth optimization problems with -norm. Finally, the given numerical experiments show the efficiency of the proposed method.

#### 1. Introduction

We consider the following unconstrained optimization problem:where is a continuous function. It is well-known that the nonlinear conjugate gradient method is one of the most effective methods for solving large-scale unconstrained optimization problems due to its simplicity and low storage [1–8]. Let be the initial approximation of the solution to (1); the general format of the nonlinear conjugate gradient method is as follows:where can be obtained by some linear searches, i.e., [6–8], and search direction is computed bywhere is the gradient of at point and is a parameter. Different choices for the parameter correspond to different nonlinear conjugate gradient methods. The Fletcher-Reeves (FR) method, the Polak-Ribiere-Polyak (PRP) method, the Hestenes-Stiefel (HS) method, the Dai-Yuan (DY) method, and the Conjugate Descent (CD) method are some famous nonlinear conjugate gradient methods [1, 2, 9–12], and the parameters of them are, respectively, defined bywhere and is the Euclidean norm. Because of the good numerical performance of the conjugate gradient method, in recent years, the nonlinear three-term conjugate gradient method has been paid much attention by researchers, such as the three-term conjugate gradient method [5], the three-term form of the L-BFGS method [13], the three-term PRP conjugate gradient method [14], and the new-type conjugate gradient update parameter similar with [15]. On the other hand, we know that the Armijo line search is widely used in solving optimization problems; i.e., see [8]. So, in this paper, we propose a new modified three-term conjugate gradient method with the Armijo line search. The proposed method is used to solve -tensor systems [16, 17] and a kind of nonsmooth optimization problems with -norm [18–22].

The remainder of this paper is organized as follows: In the next section, we give the new modified three-term conjugate gradient method. Firstly, we give the smooth case of the proposed method and prove the sufficient descent property and the global convergence property of it. Then, we give the nonsmooth case of the proposed method. In Section 3, we present -tensor systems and a kind of nonsmooth minimization problems with -norm, which can be solved by the proposed method. And, we also give some numerical results to show the efficiency of the proposed method. In Section 4, we give the conclusion of this paper.

#### 2. Modified Three-Term Conjugate Gradient Method

In this section, we consider the nonlinear conjugate gradient method for solving (1); we discuss the problem in two cases: (1) is a smooth function; (2) is a nonsmooth function.

##### 2.1. Smooth Case

Based on nonlinear conjugate gradient methods in [5, 8], we propose a modified three-term conjugate gradient method with the Armijo line search. We consider the search directionwhere is the gradient of at , , andFrom (5), (6), and (7), we can obtain that

Now, we present the modified three-term conjugate gradient method.

*Algorithm 1 (modified three-term conjugate gradient method). **Step 0*. Choose and give an initial point , let , compute , and let .*Step 1*. If , stop; otherwise, go to Step 2.*Step 2*. Compute the search direction by (5), where and are defined by (6) and (7).*Step 3*. Compute by the Armijo line search, where and satisfies*Step 4*. Compute , where is given in Step 2 and is given in Step 3.*Step 5*. Set and go to Step 1.

Next, we will give the global convergence analysis of Algorithm 1. Firstly, we give the following assumptions.

*Assumption 2. *The level set is bounded; i.e., there exists a positive constant such that for all .

*Assumption 3. *In the neighborhood of , is continuously differentiable and its gradient is Lipschitz continuous; that is, there exists a positive constant , , such that

*Remark 4. *Because is a decreasing sequence, so the sequence generated by Algorithm 1 is contained in . And by Assumptions 2 and 3, we can easily obtain that there exists a positive constant such that

Lemma 5. *Suppose and are generated by Algorithm 1, then*

*Proof. *Firstly, we prove that there exists a constant such that, for sufficiently large ,The proof of (13) can be divided into two following cases.*Case 1* (). By (8) and the Cauchy inequality, , then we have . Let , then we obtain (13).*Case 2* (). Due to the linear search step, that is the Step 3 of Algorithm 1, does not satisfy (9); i.e.,By Assumption 3 and the mean value theorem, there exists such that By the above formula, (8) and (14), we haveLet , then we obtain (13).

By (9) and Assumption 2, we haveFrom (8), (13), and (17), we have then we get Hence, the result follows.

Now we can get the global convergence of Algorithm 1.

Theorem 6. *Suppose and are generated by Algorithm 1, then*

*Proof. *Using the technique similar to Theorem 3.1 in [5], we can get this theorem.

*Remark 7. *The Armijo type line search [7] is given as follows:where , , , and The Wolfe type line search [6] is given as follows:where Obviously, Algorithm 1 is also true for the Armijo type line search and the Wolfe type line search.

##### 2.2. Nonsmooth Case

In this subsection, by using smoothing function, we extend Algorithm 1 to the nonsmooth case. Firstly, we give the definition of smoothing function.

*Definition 8. *Let be a local Lipschitz continuous function. If , , and is fixed, is continuously differentiable and satisfiesthen we call is a smoothing function of .

Denote . Now, we present the following smoothing modified three-term conjugate gradient method.

*Algorithm 9 (smoothing modified three-term conjugate gradient method). **Step 0*. Choose and give an initial point , let , compute , and let .*Step 1*. If , stop; otherwise, go to Step 2.*Step 2*. Compute the search direction by using and , wherewhere .*Step 3*. Compute by the Armijo line search, where and satisfies*Step 4*. Compute , if , set ; otherwise, let .*Step 5*. Set and go to Step 1.

Next, we give the global convergence analysis of Algorithm 9.

Theorem 10. *Suppose that is a smoothing function of . If for every fixed , satisfies Assumptions 2 and 3, then generated by Algorithm 9 satisfies*

*Proof. *Denote . If is finite, then there exists an integer such that, for all ,and . That is to solveHence, from Theorem 6, we getwhich contradicts with (27). This shows that must be infinite and . Since is infinite, we can assume that with . Then we have

#### 3. Applications

In this section, the applications of the proposed modified three-term conjugate gradient method are given. The conjugate gradient method is suitable for solving unconstrained optimization problems. In the first subsection, we consider the -tensor systems, which can be transformed into the unconstrained minimization problem and solved by Algorithm 1. Then in the second subsection, we consider a kind of nonsmooth optimization problems with -norm, which can be solved by Algorithm 9. And in each subsection, the numerical results are given to show the feasibility of the proposed method.

##### 3.1. Applications in Solving -Tensor Systems

In this subsection, we consider the -tensor systems, which can be transformed into the general unconstrained minimization problem. We use Algorithm 1 to solve it. The problem of tensor systems [16, 17] is an important problem in tensor optimization [23–26]. We consider the tensor systemwhere and . Then the th element of (31) is defined asAnd if and satisfywherethen we call is an eigenvalue of and is a corresponding eigenvector of [25]. The spectral radius [26] of a tensor is defined asLet be the identity tensor [17], i.e.,for all . If there exists a nonnegative tensor and a positive real number such that , then the tensor is called an -tensor [16]. And if , it is called a nonsingular -tensor. Suppose is a nonsingular -tensor, then for every positive vector , (31) has a unique positive solution [16]. Then (31) can be transformed into the following unconstrained minimization problem

Now, we present numerical experiments for solving -tensor systems. Some examples are taken from [16]. We implement Algorithm 1 with the codes in Matlab Version R2014a and Tensor Toolbox Version 2.6 on a laptop with an Intel(R) Core(TM) i5-2520M CPU(2.50GHz) and RAM of 4.00GB. The parameters involved in the algorithm are taken as .

*Example 11. *Consider (31) with a 3rd-order 2-dimensional -tensor, where . And contains the entries with and , and other entries are zeros. Let , . Hence is a upper triangular nonsingular -tensor. The starting point is set to be and is set to be .

The numerical results are given in Table 1 and Figure 1.