A Newton-Like Trust Region Method for Large-Scale Unconstrained Nonconvex Minimization

Abstract

We present a new Newton-like method for large-scale unconstrained nonconvex minimization. And a new straightforward limited memory quasi-Newton updating based on the modified quasi-Newton equation is deduced to construct the trust region subproblem, in which the information of both the function value and gradient is used to construct approximate Hessian. The global convergence of the algorithm is proved. Numerical results indicate that the proposed method is competitive and efficient on some classical large-scale nonconvex test problems.

1. Introduction

We consider the following unconstrained optimization: min_{𝑥∈𝑅𝑛}𝑓(𝑥), (1) where 𝑓∶𝑅^𝑛→𝑅 is continuously differentiable.

Trust region methods [1–14] are robust, can be applied to ill-conditioned problems, and have strong global convergence properties. Another advantage of trust region methods is that there is no need to require the approximate Hessian of the trust region subproblem to be positive definite. So, trust region methods are important and efficient for nonconvex optimization problems [6–8, 10, 12, 14]. For a given iterate 𝑥_𝑘∈𝑅^𝑛, the main computation of trust region algorithms is solving the following quadratic subproblem: min_{𝑠∈𝑅𝑛}𝜙_𝑘(𝑠)=𝑔^𝑇_𝑘1𝑠+2𝑠^𝑇𝐵_𝑘‖𝑠‖𝑠,s.t.≤Δ_𝑘, (2) where 𝑔_𝑘=∇𝑓(𝑥_𝑘) is the gradient of 𝑓(𝑥) at 𝑥_𝑘, 𝐵_𝑘 is the true Hessian ∇²𝑓(𝑥_𝑘) or its approximation, Δ_𝑘>0 is a trust region radius, and ‖⋅‖ refers to the Euclidean norm on 𝑅^𝑛. For a trial step 𝑠_𝑘, which is generated by solving the subproblem (2), adequacy of the predicted reduction and true variation of the objective function is measured by means of the ratio 𝑟_𝑘=𝑓𝑥_𝑘−𝑓𝑥_𝑘+𝑠_𝑘𝜙_𝑘(0)−𝜙_𝑘𝑠_𝑘. (3) Then the trust region radius Δ_𝑘 is updated according to the value of 𝑟_𝑘. Trust region methods ensure that at least a Cauchy (steepest descent-like) decrease on each iteration satisfies an evaluation complexity bound of the same order under identical conditions [11]. It follows that Newton’s method globalized by trust region regularization satisfies the same 𝑂(𝜀⁻²) evaluation upper bound; such a bound can also be shown to be tight [12] provided additionally that the Hessian on the path of the iterates for which pure Newton steps are taken is Lipschitz continuous.

Newton’s method has been efficiently safeguarded to ensure its global convergence to first- and even second-order critical points, in the presence of local nonconvexity of the objective using line search [3], trust region [4], or other regularization techniques [9, 13]. Many variants of these globalization techniques have been proposed. These generally retain fast local convergence under some nondegeneracy assumptions, are often suitable when solving large-scale problems, and sometimes allow approximate rather than true Hessians to be employed. Solving-large scale problems needs expensive computation and storage. So many researchers have studied the limited memory techniques [15–24]. The limited memory techniques are firstly applied to line search method. Liu and Nocedal [15, 16] proposed a limited memory BFGS method (L-BFGS) for solving unconstrained optimization and proved its global convergence. Byrd et al. [17] gave the compact representations of the limited memory BFGS and SR1 formula, which made it possible for combining limited memory techniques with trust region method. Considering that the L-BFGS updating formula used the gradient information merely and ignored the available function value information, Yang and Xu [19] deduced modified quasi-Newton formula with limited memory compact representation based on the modified quasi-Newton equation with a vector parameter [18]. Recently, some researchers combined the limited memory techniques with trust region method for solving large-scale unconstrained and constrained optimizations [20–24].

In this paper, we deduce a new straightforward limited memory quasi-Newton updating based on the modified quasi-Newton equation, which uses both available gradient and function value information, to construct the trust region subproblem. Then the corresponding trust region method is proposed for large-scale unconstrained nonconvex minimization. The global convergence of the new algorithm is proved under some appropriate conditions.

The rest of the paper is organized as follows. In the next section, we deduce a new straightforward limited memory quasi-Newton updating. In Section 3, a Newton-like trust region method for large-scale unconstrained nonconvex minimization is proposed and the convergence property is proved under some reasonable assumptions. Some numerical results are given in Section 4.

2. The Modified Limited Memory Quasi-Newton Formula

In this section, we deduce a straightforward limited memory quasi-Newton updating based on the modified quasi-Newton equation, which employs both the gradients and function values to construct the approximate Hessian and is a compensation for the missing data in limited memory techniques. And then we apply the derived formula in trust region method.

Consider the following modified quasi-Newton equation [18]: 𝐵_𝑘+1𝑠_𝑘=𝑦_𝑘, (4) where 𝑠_𝑘=𝑥_𝑘+1−𝑥_𝑘, 𝑦_𝑘=𝑔_𝑘+1−𝑔_𝑘, 𝑦_𝑘=(1+(𝜃_𝑘/𝑠^𝑇_𝑘𝑦_𝑘))𝑦_𝑘=𝜆_𝑘𝑦_𝑘, and 𝜃_𝑘=6(𝑓(𝑥_𝑘)−𝑓(𝑥_𝑘+1))+3(𝑔_𝑘+𝑔_𝑘+1)^𝑇𝑠_𝑘. The quasi-Newton updating matrix constructed by (4) achieves a higher order accuracy in approximating Hessian. Based on (4), the modified BFGS (MBFGS) updating is as follows: 𝐵_𝑘+1=𝐵_𝑘+𝑦_𝑘𝑦^𝑇_𝑘𝑦^𝑇_𝑘𝑠_𝑘−𝐵_𝑘𝑠_𝑘𝐵_𝑘𝑠_𝑘^𝑇𝑠^𝑇_𝑘𝐵_𝑘𝑠_𝑘=𝐵_𝑘+𝜆_𝑘𝑦_𝑘𝑦^𝑇_𝑘𝑦^𝑇_𝑘𝑠_𝑘−𝐵_𝑘𝑠_𝑘𝐵_𝑘𝑠_𝑘^𝑇𝑠^𝑇_𝑘𝐵_𝑘𝑠_𝑘. (5) For twice continuously differentiable function, if 𝑥_𝑘 converges to a point 𝑥^∗ at which 𝑔(𝑥^∗)=0 and ∇²𝑓(𝑥^∗) is positive definite, then lim_𝑘→∞𝜃_𝑘=0, and then lim_𝑘→∞𝜆_𝑘=1. Moreover, if 𝑘 is sufficiently large, the MBFGS updating approaches to the BFGS updating.

Then formula (5) can be rewritten into the straightforward formula 𝐵_𝑘+1=𝐵_𝑘−𝑎_𝑘𝑎^𝑇_𝑘+𝑏_𝑘𝑏^𝑇_𝑘, (6) where 𝑎_𝑘=𝐵_𝑘𝑠_𝑘/(𝑠^𝑇_𝑘𝐵_𝑘𝑠_𝑘)^1/2 and 𝑏_𝑘=(𝜆_𝑘/𝑦^𝑇_𝑘𝑠_𝑘)^1/2𝑦_𝑘. Thus, 𝐵_𝑘 can be recursively expressed as 𝐵_𝑘=𝐵₀+^𝑘−1_𝑖=0𝑏_𝑖𝑏^𝑇_𝑖−𝑎_𝑖𝑎^𝑇_𝑖=𝐵₀+𝑏₀,𝑏₁,…,𝑏_𝑘−1𝑏^𝑇₀𝑏^𝑇₁⋮𝑏^𝑇_𝑘−1−𝑎₀,𝑎₁,…,𝑎_𝑘−1𝑎^𝑇₀𝑎^𝑇₁⋮𝑎^𝑇_𝑘−1. (7) Let 𝑌_𝑘=[𝑏₀,𝑏₁,…,𝑏_𝑘−1], and let 𝑆_𝑘=[𝑎₀,𝑎₁,…,𝑎_𝑘−1]. Then the above formula can be simply written as 𝐵_𝑘=𝐵₀+𝑌_𝑘𝑌^𝑇_𝑘−𝑆_𝑘𝑆^𝑇_𝑘. (8) Formula (8) is called the whole memory quasi-Newton formula. For a given positive integer 𝑚 (𝑚 usually is taken for 3,5,7,…), if we use the last 𝑚 pairs (𝑠_𝑘−𝑚,𝑦_𝑘−𝑚),…,(𝑠_𝑘−1,𝑦_𝑘−1) at the 𝑘th (𝑘≥𝑚) iteration to update the starting matrix 𝐵_𝑘⁽⁰⁾𝑚 times, according to (8), we get the following limited memory MBFGS (L-MBFGS) formula: 𝐵_𝑘=𝐵_𝑘^(𝑚)=𝐵_𝑘⁽⁰⁾+𝑌_𝑘𝑌^𝑇_𝑘−𝑆_𝑘𝑆^𝑇_𝑘, (9) where 𝑌_𝑘=[𝑏_𝑘−𝑚,…,𝑏_𝑘−1], 𝑆_𝑘=[𝑎_𝑘−𝑚,…,𝑎_𝑘−1]; then 𝐵_𝑘=𝐵_𝑘⁽⁰⁾+𝑏_𝑘−𝑚,…,𝑏_𝑘−1𝑏^𝑇_𝑘−𝑚⋮𝑏^𝑇_𝑘−1−𝑎_𝑘−𝑚,…,𝑎_𝑘−1𝑎^𝑇_𝑘−𝑚⋮𝑎^𝑇_𝑘−1, (10) where 𝑎_{𝑘−𝑚+𝑗}=𝐵_{𝑘−𝑚+𝑗}𝑠_{𝑘−𝑚+𝑗}/(𝑠^𝑇_{𝑘−𝑚+𝑗}𝐵_{𝑘−𝑚+𝑗}𝑠_{𝑘−𝑚+𝑗})^1/2 and 𝑏_{𝑘−𝑚+𝑗}=(𝜆_{𝑘−𝑚+𝑗}/𝑦^𝑇_{𝑘−𝑚+𝑗}𝑠_{𝑘−𝑚+𝑗})^1/2𝑦_{𝑘−𝑚+𝑗}  (𝑗=0,1,…,𝑚−1).

Since the vectors 𝑏_𝑖 and 𝑎_𝑖  (𝑖=𝑘−𝑚,…,𝑘−2) can be obtained and saved from the previous iterations, we only need to compute the vectors 𝑏_𝑘−1 and 𝑎_𝑘−1 to achieve the limited memory quasi-Newton updating matrix. Suppose 𝐵_𝑘⁽⁰⁾=𝐼, the computation of 𝑏_𝑘−1 needs 3𝑛+3 multiplications. Then we consider the computation of 𝑎_𝑘−1. If 𝐵_𝑘−1 can be saved and multiplies by 𝑠_𝑘−1 directly, the process needs 𝑛² multiplications. In this paper, we compute the product 𝐵_𝑘−1𝑠_𝑘−1 by (9). Consider 𝐵_𝑘−1𝑠_𝑘−1=𝐵⁽⁰⁾_𝑘−1𝑠_𝑘−1+𝑌_𝑘−1𝑌^𝑇_𝑘−1𝑠_𝑘−1−𝑆_𝑘−1𝑆^𝑇_𝑘−1𝑠_𝑘−1=𝐵⁽⁰⁾_𝑘−1𝑠_𝑘−1+𝑏_{𝑘−𝑚−1},…,𝑏_𝑘−2𝑏^𝑇_{𝑘−𝑚−1}𝑠_𝑘−1⋮𝑏^𝑇_𝑘−2𝑠_𝑘−1−𝑎_{𝑘−𝑚−1},…,𝑎_𝑘−2𝑎^𝑇_{𝑘−𝑚−1}𝑠_𝑘−1⋮𝑎^𝑇_𝑘−2𝑠_𝑘−1. (11) So we need 4𝑚𝑛 multiplications to achieve 𝐵_𝑘−1𝑠_𝑘−1. Let 𝑎_𝑘−1=𝐵_𝑘−1𝑠_𝑘−1; then 𝑎_𝑘−1=(𝑠^𝑇_𝑘−1𝑎_𝑘−1)^−1/2𝑎_𝑘−1. It takes 2𝑛+1 multiplications to compute 𝑎_𝑘−1. Ignoring lower order terms, it is a total of (4𝑚+5)𝑛 multiplications to obtain 𝐵_𝑘.

It is noticed that the only difference between the limited memory quasi-Newton method and the standard quasi-Newton method is in the matrix updating. Instead of storing the matrices 𝐵_𝑘, we need to store 𝑚 pairs vectors {𝑎_𝑖,𝑏_𝑖} to define 𝐵_𝑘 implicitly. The product 𝐵_𝑘𝑣 or 𝑣^𝑇𝐵_𝑘𝑣 is obtained by performing a sequence of inner products involving 𝑣 and the 𝑚 most recent vectors pairs {𝑎_𝑖,𝑏_𝑖}.

In the following, we discuss the computation of the products 𝐵_𝑘𝑣 and 𝑣^𝑇𝐵_𝑘𝑣, 𝑣∈𝑅^𝑛. As the situation of (11), we need 4𝑚𝑛 multiplications to obtain 𝐵_𝑘𝑣. If 𝐵_𝑘𝑣 has been computed, we only need to solve a vector product to obtain 𝑣^𝑇𝐵_𝑘𝑣 which needs 𝑛 multiplications. If 𝐵_𝑘𝑣 has not been computed, we compute 𝑣^𝑇𝐵_𝑘𝑣 directly by using (9). Consider 𝑣^𝑇𝐵_𝑘𝑣=𝑣^𝑇𝐵_𝑘⁽⁰⁾𝑣+𝑣^𝑇𝑌_𝑘𝑌^𝑇_𝑘𝑣−𝑣^𝑇𝑆_𝑘𝑆^𝑇_𝑘𝑣=𝑣^𝑇𝐵_𝑘⁽⁰⁾𝑣+𝑌^𝑇_𝑘𝑣^𝑇𝑌^𝑇_𝑘𝑣−𝑆^𝑇_𝑘𝑣^𝑇𝑆^𝑇_𝑘𝑣. (12) The whole computation only requires (2𝑚+1)𝑛+4𝑚 multiplications. Thus, 2𝑚𝑛 multiplications are saved in contrast to the previous method.

If we take 𝐵_𝑘⁽⁰⁾=𝛾_𝑘𝐼,  𝑣^𝑇𝑣  and  𝑌^𝑇_𝑘𝑣, 𝑆^𝑇_𝑘𝑣 have been obtained and saved from the previous iteration, from (11), there are 2𝑚+1 multiplications to compute 𝑣^𝑇𝐵_𝑘𝑣; it is a considerable improvement on computation comparing with (2𝑚+1)𝑛.

Algorithm 1. Compute and save 𝑆_𝑘, 𝑌_𝑘. For 𝑗=0,1,…,𝑚−1,

Step  1. Compute 𝑏_{𝑘−𝑚+𝑗}=(𝜆_{𝑘−𝑚+𝑗}/𝑦^𝑇_{𝑘−𝑚+𝑗}𝑠_{𝑘−𝑚+𝑗})^1/2𝑦_{𝑘−𝑚+𝑗}.

Step  2. Compute 𝑎_{𝑘−𝑚+𝑗}=𝐵_{𝑘−𝑚+𝑗}𝑠_{𝑘−𝑚+𝑗}.

Step  3. Compute (𝑠^𝑇_{𝑘−𝑚+𝑗}𝑎_{𝑘−𝑚+𝑖})^−1/2𝑎_{𝑘−𝑚+𝑖}.

Algorithm 2. Compute 𝐵_𝑘𝑣, 𝑣^𝑇𝐵_𝑘𝑣. Let 𝑥_𝑘 be the current iteration point, the vectors 𝑎_𝑘−1, 𝑏_𝑘−1,𝑔_𝑘 and matrixes 𝑆_𝑘−1, 𝑌_𝑘−1 have been obtained by the previous iteration.

Step  1. Update 𝑆_𝑘, 𝑌_𝑘.

Step  2. Compute 𝑆^𝑇_𝑘𝑣, 𝑌^𝑇_𝑘𝑣.

Step  3. Compute 𝐵_𝑘𝑣 by (11); compute 𝑣^𝑇𝐵_𝑘𝑣 by (12).

We use the form of (9) to store 𝐵_𝑘. Instead of updating 𝐵_𝑘 into 𝐵_𝑘+1, we update 𝑆_𝑘, 𝑌_𝑘 into 𝑆_𝑘+1, 𝑌_𝑘+1.

3. Newton-Like Trust Region Method

In this section, we present a Newton-like trust region method for large-scale unconstrained nonconvex minimization.

Algorithm 3. Step  0. Given 𝑥₀∈𝑅^𝑛, 𝜀>0, Δ>0, Δ₀∈(0,Δ), 𝜂∈[0,1/4), 𝐵₀∈𝑅^𝑛×𝑛 is a given matrix. Compute 𝑔₀=∇𝑓(𝑥₀); set 𝑘∶=0.
Step  1. If ‖𝑔_𝑘‖<𝜀, then stop.
Step  2. Solve the subproblem (2) to obtain 𝑠_𝑘.
Step  3. Compute 𝑟_𝑘=𝑓𝑥_𝑘−𝑓𝑥_𝑘+𝑠_𝑘𝜙_𝑘(0)−𝜙_𝑘𝑠_𝑘. (13)
Step  4. Compute 𝑥_𝑘+1=𝑥_𝑘+𝑠_𝑘,if𝑟_𝑘𝑥>𝜂,_𝑘,otherwise. (14)
Step  5. Update the trust region radius as the following: Δ_𝑘+1=14Δ_𝑘,if𝑟_𝑘<14,min2Δ_𝑘,Δ,if𝑟_𝑘>34,Δ_𝑘,otherwise. (15)
Step  6. By implementing Algorithm 1 to update 𝑆_𝑘, 𝑌_𝑘 into 𝑆_𝑘+1, 𝑌_𝑘+1 in order to update 𝐵_𝑘 into 𝐵_𝑘+1, set 𝑘∶=𝑘+1; go to Step 1.
In Step 2, using CG-Steihaug algorithm in [3] to solve the subproblem (2), the algorithm is suitable for solving large-scale unconstrained optimization. In the solving process, the products 𝐵_𝑘𝑣 and 𝑣^𝑇𝐵_𝑘𝑣 are computed by Algorithm 2. Then the whole computation of solving subproblem only requires 𝑂(𝑛) multiplications.

To give the convergence result, we need the following assumptions.

Assumption 4. (H1) The level set Ω={𝑥∣𝑓(𝑥)≤𝑓(𝑥₀)} is contained in a bounded convex set.(H2) The gradient of the objective function 𝑓(𝑥) is Lipschitz continuous in the neighborhood of 𝑥^∗; that is, there is a constant 𝐿>0 such that 𝑔(𝑥)−𝑔𝑦≤𝐿𝑥−𝑦,∀𝑥,𝑦∈𝑅^𝑛. (16)(H3) The solution 𝑠_𝑘 of the subproblem (2) satisfies 𝜙_𝑘(0)−𝜙_𝑘𝑠_𝑘≥𝜎𝑔_𝑘minΔ_𝑘,𝑔_𝑘𝐵_𝑘, (17) where 𝜎∈(0,1].(H4) The solution 𝑠_𝑘 of subproblem (2) satisfies 𝑠_𝑘≤𝛾Δ_𝑘, (18) for 𝛾≥1.

Lemma 5. Suppose that (H1) holds and 𝐵_𝑘 is positive definite; there exist constants 𝑀₂>𝑀₁≥0 such that 𝑀₁≤𝑔_𝑘+1−𝑔_𝑘^𝑇𝑥_𝑘+1−𝑥_𝑘𝑥_𝑘+1−𝑥_𝑘²≤𝑀₂,𝑀₁≤𝑔_𝑘+1−𝑔_𝑘²𝑔_𝑘+1−𝑔_𝑘^𝑇𝑥_𝑘+1−𝑥_𝑘≤𝑀₂, (19) for any 𝑥_𝑘+1,𝑥_𝑘∈Ω with 𝑥_𝑘+1≠𝑥_𝑘. Then matrices {𝐵_𝑘} are uniformly bounded.

Proof. From Taylor expansion 𝑓𝑥_𝑘+1=𝑓𝑥_𝑘+𝑔^𝑇_𝑘𝑠_𝑘+12𝑠^𝑇_𝑘∇²𝑓𝑥_𝑘+𝑡𝑠_𝑘𝑠_𝑘,𝑡∈(0,1), (20) we have 2𝑓𝑥_𝑘−𝑓𝑥_𝑘+1+𝑔^𝑇_𝑘𝑠_𝑘=𝑠^𝑇_𝑘∇²𝑓𝑥_𝑘+𝑡𝑠_𝑘𝑠_𝑘,𝑡∈(0,1). (21) Then 𝜃_𝑘=6𝑓𝑥_𝑘−𝑓𝑥_𝑘+1+3𝑔_𝑘+𝑔_𝑘+1^𝑇𝑠_𝑘≤3𝑠^𝑇_𝑘∇²𝑓𝑥_𝑘+𝑡𝑠_𝑘𝑠_𝑘−𝑔_𝑘+1−𝑔_𝑘^𝑇𝑠_𝑘. (22) From (19), we obtain that 𝜃_𝑘≤6𝑀₂𝑠_𝑘². (23) It is obvious that 𝜆_𝑘=𝜃1+^𝑘𝑠^𝑇_𝑘𝑦_𝑘≤1+6𝑀₂𝑀₁. (24) Thus, 𝑏^𝑇_𝑘𝑏_𝑘≤𝜆_𝑘𝑦²_𝑘𝑦^𝑇_𝑘𝑠_𝑘≤𝑀₂1+6𝑀₂𝑀₁. (25)
Since Tr(𝑥𝑦^𝑇)=𝑥^𝑇𝑦(𝑥,𝑦∈𝑅^𝑛), Tr(𝐴+𝐵)=Tr(𝐴)+Tr(𝐵)  (𝐴,𝐵∈𝑅^𝑛×𝑛) and from (9) (in which 𝐵_𝑘⁽⁰⁾=𝐼), we have 𝐵_𝑘=𝐵_𝑘⁽⁰⁾+𝑏_𝑘−𝑚,…,𝑏_𝑘−1𝑏^𝑇_𝑘−𝑚⋮𝑏^𝑇_𝑘−1−𝑎_𝑘−𝑚,…,𝑎_𝑘−1𝑎^𝑇_𝑘−𝑚⋮𝑎^𝑇_𝑘−1; (26) then by (25) and 𝐵_𝑘 being positive definite, we have Tr𝐵_𝑘=Tr𝐵_𝑘⁽⁰⁾+^𝑚−1_𝑗=0𝑏^𝑇_{𝑘−𝑚+𝑗}𝑏_{𝑘−𝑚+𝑗}−𝑎^𝑇_{𝑘−𝑚+𝑗}𝑎_{𝑘−𝑚+𝑗}≤Tr𝐵_𝑘⁽⁰⁾+^𝑚−1_𝑗=0𝑏^𝑇_{𝑘−𝑚+𝑗}𝑏_{𝑘−𝑚+𝑗}≤𝑛+𝑚𝑀₂+6𝑀²₂𝑀₁. (27)
By the definition of Euclidean norm: ‖𝐴‖=𝜌(𝐴^𝑇𝐴)(𝐴∈𝑅^𝑚×𝑛), when 𝐴∈𝑅^𝑛×𝑛 is a symmetric matrix, ‖𝐴‖=𝜌(𝐴). Obviously, 𝐵_𝑘 is a symmetric matrix. Suppose the eigenvalues of 𝐵_𝑘 are 0<𝜆₁≤𝜆₂≤⋯≤𝜆_𝑛; then 𝐵_𝑘=𝜆_𝑛≤^𝑛_𝑖=1𝜆_𝑖=Tr𝐵_𝑘≤𝑛+𝑚𝑀₂+6𝑀²₂𝑀₁. (28) So, 𝐵_𝑘 is uniformly bounded.

Theorem 6. Let 𝜂=0 in Algorithm 3. Suppose that Assumption 4 holds and ‖𝐵_𝑘‖≤𝛽 for some constant 𝛽. Let the sequence {𝑥_𝑘} be generated by Algorithm 3. Then one has lim_𝑘→∞inf𝑔_𝑘=0. (29)

The proof is similar to Theorem  4.7 in [3] and is omitted.

4. Numerical Results

In this section, we apply Algorithm 3 to solve nonconvex programming problems. Preliminary numerical results to illustrate the performance of Algorithm 3 are denoted by NLMTR. The contrast tests are called NTR, which is the same as NLMTR except that 𝐵_𝑘 is updated by BFGS formula. All tests are implemented by using Matlab R2008a on a PC with CPU 2.00 GHz and 2.00 GB RAM. The test problem collections for nonconvex unconstrained minimization are taken from Moré et al. in [25], the CUTEr collection [26, 27]. These problems are listed in Table 1.

Table 1 

All numerical results are listed in Table 2, in which iter stands for the number of iterations, which equals the number of gradient evaluations; nf stands for the number of objective function evaluations; Prob stands for the problem label; Dim stands for the number of variables of the tested problem; cpu denotes the CPU time for solving the problems; ‖𝑔_𝑘‖ is the terminated gradient; and 𝑓^∗ denotes the optimal value.

Table 2 

Numerical results for NLMTR and NTR.

We compare NLMTR with NTR. The trial step 𝑠_𝑘 is computed by CG-steihaug algorithm [3]. The matrix 𝐵_𝑘 of NLMTR is updated by the straightforward modified L-MBFGS formula (9). Choosing 𝜂=0.1, 𝑚=3. The matrices 𝐵_𝑘 of NTR is updated by BFGS formula in [3]. The iteration is terminated by ‖𝑔_𝑘‖≤𝜀 or ‖𝑠_𝑘‖≤𝜀, where 𝜀=10⁻⁸. The related figures are listed in Table 2.

From Table 2, we can see that for small-scale problems, the optimal values and the gradient norms of NTR are more accurate than NLMTR. For middle-scale problems, the accuracy of NTR is higher, but the cpu time of NLMTR is shorter. For large-scale problems, the cpu time of NTR is much more than NLMTR, and for some problems NTR fails, especially when 𝑛=5000. So NLMTR is suitable for solving large-scale nonconvex problems.

Acknowledgments

This work is supported in part by the NNSF (11171003) of China, the Key Project of Chinese Ministry of Education (no. 211039), and Natural Science Foundation of Jilin Province of China (no. 201215102).