#### Abstract

A new self-adaptive rule of trust region radius is introduced, which is given by a piecewise function on the ratio between the actual and predicted reductions of the objective function. A self-adaptive trust region method for unconstrained optimization problems is presented. The convergence properties of the method are established under reasonable assumptions. Preliminary numerical results show that the new method is significant and robust for solving unconstrained optimization problems.

#### 1. Introduction

Consider the following unconstrained optimization problem: where is continuously differentiable.

The trust region methods calculate a trial step by solving the subproblem at each iteration, where and is symmetric matric approximating the Hessian of at , and is the current trust region radius. Throughout this paper, denotes the -norm. We define the ratio, and the numerator and the denominator are called the actual reduction and the predicted reduction, respectively. For basic trust region (BTR) method, if the sufficient reduction predicted by the model is realized by the objective function, the trial point is accepted as the next iterate and the trust region is expanded or kept the same. If the model reduction turns out to be a poor predictor of the actual behavior of the objective function, the trial point is rejected and the trust region is contracted, with the hope that the model provides a better prediction in the smaller region. More formally, basic trust region radius update rule can be usually summarized as follows: where the constants ,,, and satisfy

The iteration is said to be successful if . If not, the iteration is unsuccessful, and the trial point is rejected. If , the iteration is said to be very successful iteration . If is significantly larger than one, that is, , the iteration is called too successful iteration .

In our opinion, the agreement between the model and the objective function is not good enough at too successful iterations. We take that the updated trust region radius is less than and is bounded lower away from zero. It implies that is not always necessary. This strategy can also match the convergence criteria presented by Conn et al. .

In the paper, the -function is introduced, which is a variant of -function. A new self-adaptive trust region method is proposed, in which the trust region radius is . The new method is more efficient at too successful iterations.

The rest of the paper is organized as follows. In Section 2, we define -function to introduce new update rules and a new self-adaptive trust region algorithm is presented. In Section 3, the convergence properties of proposed algorithm are investigated. In Section 4, numerical results are given. In Section 5 conclusions are summarized.

#### 2. -Function and the New Self-Adaptive Trust Region Algorithm

To obtain the new trust region radius update rules, we define -function ,.

Definition 1. A function is called an -function if it satisfies the following:(1) is nondecreasing in and nonincreasing in , , for ,(2),(3),(4),(5),(6), for ,(7)and ,
where the constants ,,,,,, are positive constants such that

It is easy to prove that the -function is a bounded function in . In the following, we show the differences between the usual empirical rule, the -function rule, the -function rule, and the -function rule.

The usual empirical rules ([1, 20, 21]) (Figure 1(a)) can be usually summarized as follows: where ,,, and are predefined constants such that

The -function rule and the -function rule can be described as follows: where the -function (Figure 1(b)) proposed by Hei  is chosen aswhere , , , , and are constants, and the -function (Figure 1(c)) proposed by Walmag and Delhez  is chosen as where ,,, and are constants.

The -function rule can be described as follows: where the -function (Figure 2) is chosen as

where , , , , , and are constants.

The -function generalizes the -function and the -function. It contains some favorable features of the -function  and the -function .

Now describe the new self-adaptive trust region algorithm with improved update rules.

Algorithm 2. One has the following.
Step 1. Given , , ,,, and ; ; set .
Step 2. If or , stop. Otherwise solve subproblem (2) to get .
Step 3. Compute
Compute as follows:
Update the trust region radius where is defined by (15).

Step 4. Compute and ; set ; go to step 2.

#### 3. Convergence of Algorithm 2

In the section, we investigate the convergence properties of Algorithm 2. Since it can be considered as a variant of the basic trust region method of Conn et al. , we expect similar results and significant similarities in their proofs under the following assumptions.

Assumption 3. Consider the following.(i)The sequence is uniformly bounded in norm; that is , for some constant .(ii)The function is bounded on the level set .(iii)The solution of the subproblem (2) satisfies where .(iv)The solution of the subproblem (2) satisfies for positive constant .

Lemma 4. Suppose that Assumption 3 holds. Then where arbitrarily decreases with decreasing.

Proof. Since from Taylor theorem, we have that it follows from the definition of in (2) that

By Algorithm 2, we are capable of showing that the iteration must be very successful but not too successful if the trust region radius is sufficiently small enough and also that the trust region radius has to increase in this case. The following lemma’s proof is a bit different from the proof of Theorem in .

Lemma 5. Suppose that Assumption 3 holds. If and there exists a constant such that then

Proof . Using Assumption 3 and and assuming that there is such that , we obtain that Combining (21) and (26), we have By (24), we can choose sufficient small , such that
Using (27) and (28), we have ; that is, . Since , then . And so, by Algorithm 2, we have , where falls below the threshold .

The proof of Lemma 5 efficiently uses the conditions to explain the case of too successful iteration, as distinguished from the proof of Theorem in .

Theorem 6. Suppose that Assumption 3 holds. Let the sequence be generated by Algorithm 2. Then

Proof. Assume, for the purpose of deriving a contradiction, that, for all ,
Suppose that there is an infinite iteration subsequence such that . Using Algorithm 2 and (21), we have where . Since is bounded below, then which contradicts (25). Hence (30) holds.

In our strategy of the trust region radius’ adjustment, most of the iterations are indeed very successful but not too successful; the trust region constraint becomes irrelevant in the local subproblem. Therefore, superlinear convergence of trust region algorithm is preserved by the proposed self-adaptive radius update.

#### 4. Numerical Experiments

In this section, we present preliminary numerical results to illustrate that the performance of Algorithm 2, denoted by LATR, the basic trust region method in , denoted by BTR, and the parameters needed in (9) are chosen that ,,, and ; the adaptive trust region method in , denoted by RATR, and the parameters needed in (12) are chosen that ,,, and ; and the adaptive trust region method in , denoted by ATR, and the parameters needed in (12) are chosen that ,,, and . In Algorithm 2, , , , , , , and . All tests are implemented by using MATLAB R2012a on a PC with CPU 2.67 GHz and 8.00 GB RAM. The first eleven problems are taken from ; others are from the CUTEr collection [15, 23]. The discrete Newton method is used to update the approximate Hessian matrix . To stabilize the algorithms, the approximate Hessian matrix can be chosen as follows: where is obtained by forward difference formula at (see ) and is an identity matrix.

In all trust region algorithms in this paper, the trial step is computed by CG-Steihaug algorithm in . The iteration is terminated by the following condition: except for problem Waston, which will exceed 500 iterations. For the problem, the stopping criterion is In Table 1, we give the dimension (Dim) of each test problem, the number of function evaluations (nf), and the number of gradient evaluations (ng). In many cases, algorithm LATR is superior than algorithms BTR, RATR, and ATR, especially for solving problems (1), (7), (8), (13), (19), and (28), although the numbers of gradient evaluations are a bit more than others sometimes. Further result is shown by Figure 3, which is characterized by means of performance profile proposed in . Consider the following performance profile function: where and is the performance ratio of a solver on a problem; denotes the number of the tested problems, the number of the solvers, and the number of the function evaluations (or the CPU time, the number of gradient evaluations, number of iterations, etc.) required to solve the problem by the solver .

From Table 1, we know that and ; then performance profile is given on the sum of the number of function and gradient evaluations to solve the problem. As we can see on Figure 3, the new self-adaptive algorithm is superior to the other three algorithms.

#### 5. Conclusion

This paper presents a new self-adaptive trust region algorithm according to the new self-adaptive radius update rule. As the iteration istoo successful, we suggest reducing the trust region radius with the new rules. The convergence properties of the method are established under reasonable assumptions. Numerical experiments show that the new algorithm for solving unconstrained optimization problems is significant and robust.

For future research, we should investigate how to select an appropriate -function to conduct numerical experiments.

#### Conflict of Interests

The authors declare that they have no financial nor personal relationships with other people or organizations that can inappropriately influence their work; there is no professional nor another personal interest of any nature or kind in any product, service, and/or company that could be construed as influencing the position presented in or the review of the paper.

#### Acknowledgments

This research is partly supported by Chinese NSF under Grant 11171003, Chinese Ministry of Education, Science and Technology Research Key Project no. 2011039, and Department of Education of Jilin Province Project no. 2009158 and no. 201215102.