Abstract
We introduce a new parametric kernel function, which is a combination of the classic kernel function and a trigonometric barrier term, and present various properties of this new kernel function. A class of large- and small-update primal-dual interior-point methods for linear optimization based on this parametric kernel function is proposed. By utilizing the feature of the parametric kernel function, we derive the iteration bounds for large-update methods, , and small-update methods, . These results match the currently best known iteration bounds for large- and small-update methods based on the trigonometric kernel functions.
1. Introduction
In this paper, we consider the linear optimization (LO) problem in standard form where with rank, , and . The dual problem of is given by
For years, LO has been one of the most active research areas in mathematical programming. There are many solution approaches for LO. Among them, the interior-point methods (IPMs) gain much more attention. Several efficient IPMs for LO and a large amount of results have been proposed. For an overview of the relevant results, see a recent book on this subject [1] and the references cited therein.
In the literature two types of primal-dual IPMs are distinguished: large-update methods and small-update methods, according to the value of the barrier-update parameter . However, there is still a gap between the practical behavior of these algorithms and these theoretical performance results. The so-called large-update IPMs have superior practical performance but with relatively weak theoretical results. While the so-called small-update IPMs enjoy the best known worst-case iteration bounds but their performance in computational practice is poor.
Recently, this gap was reduced by Peng et al. [2] who introduced the so-called self-regular kernel functions and designed primal-dual IPMs based on self-regular proximities for LO. They improved the iteration bound for large-update methods from to , which almost closes the gap between the iteration bounds for large- and small-update methods. Later, Bai, et al. [3] presented a large class of eligible kernel functions, which is fairly general and includes the classical logarithmic function and the self-regular functions, as well as many non-self-regular functions as special cases. The best known iteration bounds for LO obtained are as good as the ones in [2] for appropriate choices of the eligible kernel functions. Some well-known eligible kernel functions and the corresponding iteration bounds for large- and small-update methods are collected in Table 1. For some other related kernel-function based IPMs we refer to the recent books on this subject [4, 5].
Particularly, El Ghami et al. [6] first introduced a trigonometric kernel function for primal-dual IPMs in LO. They established the worst case iteration bounds for large- and small-update methods, namely, and , respectively. Peyghami et al. [7] considered a new kernel function with a trigonometric barrier term. Based on this kernel function, they proved that large-update method for solving LO has the worst case iteration bound, namely, , which improves the so far obtained iteration bound for large-update methods based on the trigonometric kernel function proposed in [6]. Recently, Peyghami and Hafshejani [8] established the better iteration bound for large-update methods based on a new kernel function consisting of a trigonometric function in its barrier term.
Motivated by their work, the purpose of this paper is to deal with the so-called primal-dual IPMs for LO based on a new kind of parametric kernel function as follows: where (the bound of the is due to the proof of Lemma 3) and . We develop some new properties of the parametric kernel function, as well as the corresponding barrier function. Compared to the existing ones, the proposed function has a parameter . This implies that our kernel function includes a class of kernel functions. Furthermore, we present a class of primal-dual IPMs for LO based on this new parametric kernel function. The obtained iteration bound for large-update methods, namely, , which improves the classical iteration complexity with a factor , and for small-update methods, we derive the iteration bound, namely, , which matches the currently best known iteration bound for small-update methods.
The paper is organized as follows. In Section 2, we present the framework of kernel-based IPMs for LO. In Section 3, we introduce the new parametric kernel function with a trigonometric barrier term and develop some useful properties of the new kernel function, as well as the corresponding barrier function. The analysis and complexity of the algorithms for large- and small-update methods are presented in Section 4. Finally, Section 5 contains some conclusions and remarks.
Some notations used throughout the paper are as follows. , , and denote the set of vectors with components, the set of nonnegative vectors, and the set of positive vectors, respectively. denotes the 2-norm of the vector . denotes the identity vector. For any , (or ) denotes the smallest (or largest) value of the components of . Finally, if is a real valued function of a real nonnegative variable, the notation means that for some positive constant and means that for two positive constants and .
2. Framework of Kernel-Based IPMs for LO
In this section, we briefly recall the framework of kernel-based IPMs for LO, which includes the central path, the new search directions, and the generic primal-dual interior-point algorithm for LO.
2.1. Central Path for LO
Throughout the paper, we assume that both and satisfy the interior-point condition (IPC); that is, there exists such that
The Karush-Kuhn-Tucker (KKT) conditions for and are given by where denotes the component-wise product of the vectors of and . The standard approach is to replace the third equation in (3), the so-called complementarity condition for and , by the parameterized equation , with . This yields the following system:
Since and the IPC holds, the parameterized system (4) has a unique solution for each . This solution is denoted as and we call the -center of and the -center of . The set of -centers (with running through all positive real numbers) gives a homotopy path, which is called the central path of and . If , then the limit of the central path exists and since the limit points satisfy the complementarity condition (3), the limit yields optimal solutions for and (see, e.g., [1]).
2.2. New Search Directions
IPMs follow the central path approximately and approach the optimal set of LO by letting go to zero. Applying Newton’s method to the system (4), we have
This system has a unique solution [2, 3]. Defining the vector
Note that the triple coincides with the -center if and only if . For further use we introduce the scaled search directions and according to
By using (6) and (7), after some elementary reductions, we have where with .
It is obvious that the right-hand side in the third equation of the system (8) equals minus the derivative of the classic barrier function as follows: where is the kernel function of the classic barrier function. Thus, the system (8) can be rewritten as the following system:
Corresponding to the parametric kernel function (1), we define the barrier function as follows:
Due to the properties of the parametric kernel function , see, for example, Section 3, we can conclude that is a strictly convex function and attains minimal value at and ; that is,
Hence, the value of can be considered as a measure of the distance between the given iterate and the -center of the algorithms.
The approach in this paper differs only in one detail: we replace the right-hand side of the third equation in (8) by . This yields the following system:
The scaled search directions and are orthogonal vectors due to the fact that belongs to the null space and to the row space of the matrix . From (13), one may easily verify that the right-hand side in the system (14) vanishes if and only if . Thus we conclude that , , and all vanish if and only if , that is, if and only if , , and . Otherwise, we will use as the new search direction.
For the analysis of the interior-point algorithm, we define the norm-based proximity measure as follows:
One can easily verify that
2.3. Generic Primal-Dual Algorithm for LO
In general each kernel function gives rise to a primal-dual interior-point algorithm. Without loss of generality we assume that is known for some positive . For example, due to the above assumption we may assume this for , with . Then, we decrease to for some . We solve the scaled Newton system (14) and through (7) to get the new search direction . The new triple is given by where denotes the default step size, , which has to be chosen appropriately. If necessary, we repeat the procedure until we find iterates that are in the neighborhood of . Then is again reduced by the factor and we apply Newton’s method targeting the new -centers, and so on. This process is repeated until is small enough, say until ; at this stage we have found an -solution of and . The generic form of this algorithm is shown in Algorithm 1.
| ||||||||||||||||||||||||||||||||||
3. New Parametric Kernel Function and Its Properties
In this section, we introduce the new parametric kernel function with a trigonometric barrier term and develop some useful properties of the new kernel function as well as the corresponding barrier function that are needed in the analysis of the algorithms.
For ease of reference, we give the first three derivatives of given by (1) with respect to as follows: where
One can easily verify that
This implies that the kernel function is completely defined by its second derivative as follows:
In what follows, we develop some technical lemmas on the parametric kernel function.
Lemma 1. Let . Then
Proof. Define . For , we have
It follows that
Thus is monotone decreasing in , and since , this implies the lemma.
Lemma 2. Let be a constant, and Here are the functions of parameter for . If , and for , then we have for all .
Proof. It is obvious that , for all . This implies that is monotone increasing. Since , we have for all . And so on, we can conclude that for all . This completes the proof of the lemma.
Lemma 3. Let . Then
Proof. Firstly, we consider two cases to prove (28).
Case 1. Assume that . Then we have . Since , from (19) one can see that for all , when .
Case 2. Assume that . Define
We need to prove that when , holds. Using the fact that for all , we have
where
From Lemma 2, by solving for , we can conclude that
which implies that for all , when .
Secondly, we consider three cases to prove that when , (29) holds. We have
Case 1. Assume that . Since , we have and ; therefore holds for this case.
Case 2. Assume that . ; therefore also holds for this case.
Case 3. Assume that . Using Lemma 1, we have
This implies that for all .
From three cases above we conclude that when , we have for all .
Thirdly, we consider two cases to prove that when , (30) holds.
Case 1. Assume that . We have and ; therefore holds for this case.
Case 2. Assume that . Then using the fact that and , we have
where
From Lemma 2, by solving for , we can conclude that
which implies that for all , when .
Finally, we consider three cases to prove that (31) holds while . We have
Define
Case 1. Assume that . In this situation, we have , , , and , from which we know that . Therefore holds for this case.
Case 2. Assume that . Due to the fact that , , and for all , we have
where
From Lemma 2, by solving for , we can conclude that
which implies that for all , when .
Case 3. Assume that . Due to the fact that , , and for all , we have
where
From Lemma 2, by solving for , we can conclude that
this implies that when , we have for all .
From the above discussions, the proof of the lemma is completed.
The property described below is exponential convexity, which has been proven to be very useful in the analysis of primal-dual interior-point algorithms based on the eligible kernel functions [2, 3].
Lemma 4. Let . Then
Proof. The result of the lemma follows immediately from Lemma 1 in [2], which states that the above inequality holds if and only if for all . Hence, from (29) in Lemma 3, the proof of the lemma is completed.
From (28) of Lemma 3 (i.e., ), we say that is strongly convex. The following lemma provides an important consequence of this property. These results can be directly obtained from the corresponding results in [3].
Lemma 5. Let . Then
As the consequences of Lemma 5, one has the following two corollaries.
Corollary 6. Let . Then
Corollary 7. Let . Then
Lemma 8. Let . Then
Proof. Let
Then
As , to prove the lemma, it is sufficient to show that the function is a decreasing function. For this purpose, we have
If , then , so .
If , then . Using , we have
This implies the result of the lemma.
Theorem 9. Let and . Then
Proof. It follows from Lemma 8 with that
Thus, we have, by Corollary 7,
The proof of the theorem is completed.
4. Analysis and Complexity of the Algorithms
In this section, we first choose a default step size. Then, we derive an upper bound for the decrease of the barrier function during an inner iteration. Finally, the iteration bounds for large- and small-update methods are established.
4.1. Computation of the Default Step Size
In each inner iteration, we first compute the search direction from the system (14). Then through (7), we obtain the search direction . Note that during an inner iteration the parameter is fixed. Hence, after the step the new -vector is given by
It follows from (17) that
Also using , we obtain
We consider the decrease in as a function of and define
However, working with may not be easy because in general is not convex. Thus, we are searching for the convex function that is an upper bound of and whose derivatives are easier to calculate than those of .
Lemma 4 implies that
Let
This makes clear that is an upper bound of . Furthermore, we can easily verify that .
Taking the derivative with respect to , we have
This gives, also using the third expression of system (14),
Differentiating once again, we get
Below we use the shorthand notation: . The following lemma provides an upper bound of , which can be found in Lemma 4.1 in [3].
Lemma 10. One has .
Following the strategy considered in [3], we briefly recall how to choose the default step size. Suppose that the step size satisfies
Then . The largest possible value of the step size of satisfying (71) is given by where is the inverse function of for . Furthermore, we can conclude that
Since is the inverse function of , one has
This implies for . Hence, putting , we get . Thus one has
From Lemma 1, it yields which implies that
It also means that
Since and for all , one has
Therefore, using Corollary 6 (i.e., ), one has
Define
Then, we have
In the sequel, we use as the default step size, which essentially depends only on the norm and on the constant .
4.2. Decrease of the Barrier Function during an Inner Iteration
In what follows, we will show that the barrier function in each inner iteration with the default step size , as defined by (84), is decreasing. For this, we need the following technical result.
Lemma 11 (Lemma 12 in [2]). Let be a twice differentiable convex function with , and let attain its (global) minimum at . If is increasing for , then
As a consequence of Lemma 11 and the fact that , which is a twice differentiable convex function with and , we can easily prove the following lemma.
Lemma 12. Let the step size be such that . Then .
The following theorem shows that the default step size (84) yields a sufficient decrease of the barrier function during each inner iteration.
Lemma 13. Let be the default step size as given by (84). Then
Proof. From Lemma 12, (84), and Corollary 6, we have
This completes the proof of the theorem.
4.3. Iteration Bounds for Large- and Small-Update Methods
From Theorem 9, after updating the parameter to with , we have
At the start of an outer iteration and just before updating the parameter , we have . Due to (88), the value of exceeds the threshold after updating . Therefore, we need to count how many inner iterations are required to return to the situation where . We denote the value of after the -update as and the subsequent values in the same outer iteration are denoted as , , where denotes the total number of inner iterations in the outer iteration. Hence, we have
According to a decrease of in Lemma 13, we have where and .
Lemma 14 (Lemma 14 in [2]). Let be a sequence of positive numbers such that where and . Then .
The following lemma provides an estimate for the number of inner iterations between two successive barrier parameter updates, in terms of and the constant .
Lemma 15. One has
Proof. Using (90) and also applying Lemma 14, the result of the lemma follows.
The number of outer iterations is bounded above by (cf., [1, .17, page 116]). By multiplying the number of outer iterations and the number of inner iterations, we get an upper bound for the total number of iterations, namely,
Then, the iteration bound for large-update methods is established in the following theorem.
Theorem 16. For large-update methods, one takes for a constant (independent on ), namely, and . The iteration bound then becomes which improves the classical iteration bound with a factor . Similar to the analysis in [10], the iteration complexity for the small-update methods is straight and we leave it for the interested readers.
Theorem 17. For small-update methods, one takes and . The iteration bound then becomes which matches the currently best known iteration bound for small-update methods.
5. Conclusions and Remarks
In this paper, we have proposed a class of primal-dual IPMs for LO based on a new parametric kernel function, which is a combination of the classic kernel function and a trigonometric barrier term. The worst case iteration bounds for large- and small-update methods are established, namely, and , respectively. For both versions of the kernel-based IPMs, the obtained iteration bounds match the currently best known iteration bound for such methods based on the trigonometric kernel functions.
The paper improved the complexity results for large-update methods obtained by El Ghami et al. [6] and generalized the results presented recently by Peyghami et al. [7]. Furthermore, the analysis deviates significantly from the analysis presented in previous papers [6, 7].
Some interesting topics for further research remain. The extension to second-order cone optimization (SOCO), semidefinite optimization (SDO), and symmetric cone optimization (SCO) deserves to be investigated. Furthermore, the numerical results may help us compare the behavior of the proposed algorithms with other existing IPMs.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors would like to thank the anonymous referees for their useful comments and suggestions, which helped improve the presentation of this paper. This work was supported by Connotative Construction Project of Shanghai University of Engineering Science (no. NHKY-2013-08).