Abstract
We study the problem of computing weighted analytic center for system of linear matrix inequality constraints. The problem can be solved using Standard Newton’s method. However, this approach requires that a starting point in the interior point of the feasible region be given or a Phase I problem be solved. We address the problem by using Infeasible Newton’s method applied to the KKT system of equations which can be started from any point. We implement the method using backtracking line search technique and also study the effect of large weights on the method. We use numerical experiments to compare Infeasible Newton’s method with Standard Newton’s method. The results show that Infeasible Newton’s method moves in the interior of the feasible regions often very quickly, starting from any point. We recommend it as a method for finding an interior point by setting each weight to be 1. It appears to work better than Standard Newton’s method in finding the weighted analytic center when none of weights is very large relative to the other weights. However, we find that Infeasible Newton’s method is more sensitive than Standard Newton’s method to large variation in the weights.
1. Introduction
We consider a system of linear matrix inequality constraints given as follows:where is a variable and each is an symmetric matrix. Linear matrix inequality (LMI) constraints have been well studied especially in the field of semidefinite programming [1, 2]. LMI constraints have applications in a variety of fields including engineering, geometry, and statistics. We assume that feasibility determined by the constraints is bounded and has a nonempty interior. This means that the set is linearly independent [3].
In this paper, we are concerned with computing weighted analytic center for LMIs using Infeasible Newton’s method. A feasible starting point is not required to start the method. In the special case of linear constraints, weighted analytic center has been studied extensively in the past (e.g., [4]). A weighted analytic center for LMIs which extends the definition given in [4] was given in [5, 6]. The study weighted analytic center is of interest in its own right. Many algorithms for linear programming and semidefinite programming are based on weighted analytic centers [2, 3, 7, 8].
Weighted analytic center for linear matrix inequalities can be found using Standard Newton’s method by minimizing the barrier function. This approach has the drawback that a starting in the interior of the feasible region must be given. Also, Newton’s method does not work well when some of the weights are relatively very large relative to the other weights. Infeasible Newton’s method for analytic center for single LMI constraint is given in [9]. We present Infeasible Newton’s method for finding weighted analytic center that can be started from any point. The method is applied to the Karush-Kuhn-Tucker (KKT) system of equations for the weighted analytic center problem. We implement the method using backtracking line search technique and also study the effect of large weights on the method. We use numerical experiments to compare Infeasible Newton’s method with Standard Newton’s method.
We find that Infeasible Newton’s method moves very quickly into the interior of the feasible regions for most of our test problems. It seems to be a suitable method for finding an interior point for the system by setting each weight to be 1. It works better than Standard Newton’s method if none of the weights is relatively very large with respect to the other weights. We also find that Infeasible Newton’s method is more sensitive to large variations in the weights than Standard Newton’s method. In the case of very large variation in the weights, we recommend using Infeasible Newton’s method to get into the interior with each weight set to 1 and then switching to Standard Newton’s method for convergence to the weighted analytic center using the original weights and starting from the interior.
2. Weighted Analytic Center for Linear Matrix Inequalities
Let denote the feasible region defined by inequalities (1). Given , define the barrier function byThe weighted analytic center for system (1) is defined by [5, 6]
This definition extends that given in [4] for linear constraints. When , is called the analytic center. Weighted analytic center has been used in interior point methods for linear programs and semidefinite programs [2, 3, 7, 8, 10]. The primal-dual central path in semidefinite programming converges to the analytic center of the optimal solution set [11].
Standard Newton’s method has the choice for finding weighted analytic center. The gradient and Hessian of the barrier function are given by [6] the following: for Standard Newton’s Method for Computing Weighted Analytic Center Input: An interior point , tolerance Set Repeat(1)Compute the Newton’s direction (2)Compute (3)Do line search to get stepsize (4)Update (5)Update Until .
Line search technique such as backtracking line search technique can be used in Newton’s method to find weighted analytic center [9].
3. Infeasible Newton’s Method for Computing Weighted Analytic Center
In this section, we describe Infeasible Newton’s method for finding weighted analytic center. The problem of computing the weighted analytic center in (3) is a more general form of the determinant maximization problem [3]. Its dual is given by the following:where the bullet is the matrix dot-product. Theorem 1 gives optimality conditions for computing the weighted analytic center .
Theorem 1 (see [3]). Suppose both primal problem (3) and dual problem (5) are strictly feasible. The set of primal optimal solutions and dual optimal solutions is the set of feasible solutions to the system
The optimality conditions in Theorem 1 can be written equivalently asNow, as in [3], letwhere is the map that stacks the columns of a matrix on top of each other into a single vector and is the inverse map. Also, letThen, the system of (7) becomes . Newton’s directions for the system are found by solving the linear system:Using block elimination, we getwhereThe matrix is positive definite. The following is an iteration of Infeasible Newton’s method. In Step 2, the iterate is not symmetric. We symmetrize in Step 3.
An Iteration of Infeasible Newton’s Method for Computing Weighted Analytic Center
Step 1. Compute Newton’s direction using (13).
This gives .
Step 2. For each , determine
Step 3. Symmetrize .
Replace by .
Step 4. Do line search to get stepsize .
Step 5. Update the iterates
Any point can be picked as a starting point. Then, for chooseThe above iteration is repeated until , where is the residual and is a given tolerance. One can use backtracking line search [9] or other techniques to get the stepsize .
4. Numerical Experiments
In this section, we give numerical experiments to compare Infeasible Newton’s method with Standard Newton’s method. We also investigate the effects of large weights on the two methods.
Table 1 describes the 35 random test problems used for our numerical experiments. The second column of Table 1 gives the dimension of the ambient space and the third column is the number of constraints. The dimensions of the matrices are given in the fourth column. For each random problem, , , and are given and the LMI was generated randomly as follows: is an diagonal matrix with each diagonal entry chosen from . Each () is a random symmetric and sparse matrix with approximately nonzero entries generated using the MATLAB command . Each problem has a nonempty interior.
Our codes were written in MATLAB and ran on Dell OPTIPLEX 880 computer. Both Infeasible Newton’s method and Standard Newton’s method were implemented using a tolerance of and up to a maximum of 500 iterations. The starting point is random such that each of its components is chosen from a normal distribution with mean 0 and variance . We use the backtracking line search technique in the two methods. Table 2 compares Infeasible Newton’s method with Standard Newton’s method for different sets of weights. In each of Problems 1–15, one weight was set at , which is a very high value relative to the others. For Problems 16–35, none of the weights was relatively very large. In Table 2, the third and the fourth columns give the number of iterations and time required to find the first point in the interior of the feasible region (resp.) by Infeasible Newton’s method. The fourth and the fifth columns give the number of iterations and time required to find the weighted analytic center (resp.) starting from the first interior point found. The sixth and the seventh columns in Table 2 give the number of iterations and time required to find the weighted analytic center (resp.) by Standard Newton’s method. Standard Newton’s method is started from the same interior point as in Infeasible Newton’s method. Infeasible Newton’s method and Standard Newton’s method are compared in Table 3 with weights .
Figure 1 shows the iterates taken by Infeasible Newton’s method to converge in Problem 3 with . It is clear from the figure that Infeasible Newton’s method slowed down considerably before converging to the weighted analytic center. On the other hand, as seen in Figure 2, the method converged quickly with . Figure 3 shows how the norm of the gradient varies with the number of iterations for the two values of the weights. In Table 2, the entry means that Infeasible Newton’s method has failed to converge after the maximum number of 500 iterations in Problems 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 29. However, it managed to find an interior point of (1) in Problems 2, 4, 5, 7, 9, 10, 12, and 29. We see from the table that both methods might not work well when some of the weights are relatively larger than the other weights. The results also show that Standard Newton’s method performs better than Infeasible Newton’s method in this case. In Infeasible Newton’s method, the Jacobian of the residual function becomes increasingly ill conditioned near the boundary of systems (8) and (9) due to matrices and as the variation among the weights increases. Observe that Infeasible Newton’s method failed to converge in 8 problems, even though it found a point in the interior of the feasible region (1). Standard Newton’s method converged (within the 500 iterations’ limit) for each of the 8 problems except in Problem 29. Standard Newton’s method converged after 536 iterations in Problem 29.
In Table 3, the entry in Problem 29 indicates that Infeasible Newton’s method has failed to converge to the analytic center after the maximum number of 500 iterations. However, it managed to find an interior point of (1) after 1 iteration. Problem 29 shows both Infeasible Newton’s method and Standard Newton’s method may fail to converge (within 500 iterations) even if the weights are all equal and an interior point is found. We see from Table 3 that Infeasible Newton’s method took a fewer number of iterations in 12 out of 35 problems while Standard Newton’s method took a fewer number of iterations in 9 out of 35 problems. Infeasible Newton’s method took less time in 23 out of 35 problems and Standard Newton’s method took less time in 11 out of 35 problems. It is interesting to note from the table that Infeasible Newton’s method found an interior point of (1) within 1–3 iterations on most of the test problems.
The results from Tables 2 and 3 suggest that when none of the weights is relatively very large, Infeasible Newton’s method is a better method than Standard Newton’s method to find the weighted analytic center. When one weight is relatively very large, one could use Infeasible Newton’s method with to find an interior point and then switch to Standard Newton’s method using the original weights and starting from the interior point found.
5. Conclusion
We presented Infeasible Newton’s method for computing weighted analytic center for system of linear matrix inequalities and compared it with Standard Newton’s method.
We found that Infeasible Newton’s method finds a point in the interior point fairly quickly, starting from any point, especially when none of the weights is relatively very large. When none of the weights is relatively very large, Infeasible Newton’s method seems to work better than Standard Newton’s method to find the weighted analytic center. However, Infeasible Newton’s method does not work well when some of the weights are very large relative to others, but it still often finds a point in the interior, starting from any point. We find that Infeasible Newton’s method is suitable for finding an interior point for the system by setting each weight to be 1. We recommend that when some weights are relatively very large, one should use Infeasible Newton’s method to find a point in the interior with to find an interior point and then switch to Standard Newton’s method using the original weights and starting from the interior point found.
To improve the efficiency of Infeasible Newton’s method for finding weighted analytic center, it would be useful to exploit the block-diagonal structure of matrices , , and in future implementation. We would like to investigate how the method could handle weights where some are relatively much larger than the other weights. Furthermore, it would be of interest to study the application of Infeasible Newton’s method for weighted analytic center to the SDP-CUT algorithm for semidefinite programming, presented in [8]. Implementing Infeasible Newton’s method for weighted analytic center using different search directions is currently under investigation. The problem of weighted analytic center is a determinant maximization problem and can be solved with the algorithms for those problems. It would be of interest to compare the performance of those algorithms with the method presented in this paper.
Conflict of Interests
The author declares that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
The author would like to thank Jubail University College in Saudi Arabia for providing a home for the 2014-2015 sabbatical leave. This research work was done during this leave.