Novel Interior Point Algorithms for Solving Nonlinear Convex Optimization Problems
This paper proposes three numerical algorithms based on Karmarkar’s interior point technique for solving nonlinear convex programming problems subject to linear constraints. The first algorithm uses the Karmarkar idea and linearization of the objective function. The second and third algorithms are modification of the first algorithm using the Schrijver and Malek-Naseri approaches, respectively. These three novel schemes are tested against the algorithm of Kebiche-Keraghel-Yassine (KKY). It is shown that these three novel algorithms are more efficient and converge to the correct optimal solution, while the KKY algorithm fails in some cases. Numerical results are given to illustrate the performance of the proposed algorithms.
The simplex method has never had any serious competition until 1984 when Karmarkar proposed a new-polynomial-time algorithm in order to solve linear programming problems, especially for solving large scale problems [1, 2]. Polynomial complexity of Karmarkar’s algorithm has an advantage in comparison with exponential complexity of the simplex algorithm [3–5]. The improvement of the Karmarkar’s algorithm by Schrijver [6, 7] resulted in less number of iterations compared to Karmarkar’s method. In 2004, Malek and Naseri  proposed a modified technique based on the interior point algorithm to solve linear programming problems more efficiently.
After the appearance of Karmarkar’s algorithm for solving linear programming problems, the researchers developed the algorithm to solve the convex quadratic programming problem (e.g., see ). Considering the success of the interior methods for solving linear programming problems [10, 11], the researchers used the linearization methods to solve convex nonlinear programming problems. In 2007, Kebbiche et al.  introduced a projective interior point method to solve more general problems with nonlinear convex objective function. The objective of this paper is to propose an optimal step length in each iteration in combination with Karmarkar’s algorithm in order to decrease the nonlinear objective function as fast as possible.
In Section 2, extension of Karmarkar’s algorithm for nonlinear programming problem is considered. In Section 3, by considering the technique associated with KKY’s algorithm, a modified algorithm is presented. In Section 4, the convergency of modified algorithm in Section 3 is proved. Then this algorithm is combined with Schrijver’s and Malek-Naseri’s algorithms successfully. In Section 5, numerical results are illustrated for KKY algorithm and the three suggested modified algorithms are compared to each other.
2. Extension of Karmarkar’s Algorithm for Nonlinear Problems
Consider the following nonlinear optimization problem: where is a nonlinear, convex, and differentiable function. is matrix of rank and is . The starting point is chosen to be feasible. is the identity matrix with order and is a row vector of ones.
Assume , the simplex , and the projective transformation of Karmarkar defined by such that . Consider , where is a nonlinear, convex, and differentiable function and the optimal value of is zero. Using the linearization process to the function in neighborhood of the ball of center as for along with Karmarkar projective transformation , we conclude that Problem (1) is equivalent to
As a result, the optimal solution of the preceding problem lies along the negative projection of the gradient and is given as , where is the center of the simplex . is the projected gradient which can be shown to be , where , , , and is the number of iterations. The selection of as a step length is crucial to enhance the efficiency of the algorithm.
Consider the algorithm of Kebiche-Keraghel-Yassine (KKY) for solving Problem (2).
KKY Algorithm. Let be a given tolerance and let .
Step 1. Compute and . Put .
Step 2. Build and .
Compute , , and .
Step 3. While , let , and go to Step 2.
However, as we can see in the following two examples, the KKY algorithm does not work for every nonlinear problem.
Example 1. Consider the quadratic convex problem:
For tolerance , after iterations for KKY algorithm we have , , , and . Since the condition holds, the algorithm stopped, whereas this solution is not feasible.
Example 2. Consider the nonlinear convex problem: For , , and iteration using KKY algorithm the solution is calculated. This solution is not feasible.
Here, two difficulties arise. First, must stay feasible in each iteration. Second, in the moderate time the required tolerance must be satisfied. To overcome these difficulties we modify the KKY algorithm in the next section.
3. Modified Algorithm
In the KKY  algorithm . As it is observed in Examples 1 and 2, for these values of , the KKY algorithm may produce infeasible solution. In the standard Karmarkar [1, 3, 4] instead of , is used. Hence the solution in each iteration remains feasible. In this case, the optimal solution for Problem (2) is given by , where . Applying KKY algorithm with to the problem in Example 2, it gives the feasible solution , , , and . The algorithm stops after iterations since , while the suitable accuracy is not reached. In each iteration, linear search method is used to find such that , where the suitable tolerance satisfies. Thus one may write Problem (2) in the following form:
Lemma 3. The optimal solution for Problem (5) is given by .
Proof. We put , and then we have and Problem (5) is equivalent to is a solution of Problem (6) if and only if there exist and such that Multiplying both sides of (7) by and since , we get . Then ; by substituting in (7) we find By assuming , we have And we have .
Note that here we have proposed the algorithm similar to KKY, where . This modified algorithm has the advantage that can find the feasible approximate solution in the suitable tolerance.
3.2. Modified KKY Algorithm (MKKY)
Let be a given tolerance and is a strictly feasible point.
Step 1. Compute , , and . Put and .
Step 2. Build , and .
Compute , , and .
Step 3. While , put .
Build , and .
Compute and .
Let , and go to Step 3.
Step 4. While , compute , , and .
Step 5. Let , and go to Step 3.
4. Convergence for Modified Algorithm (MKKY)
In order to establish the convergence of the modified algorithm, we introduce a potential function associated with Problem (1) defined by where is the optimal value of the objective function.
Lemma 4. If is the optimal solution of Problem (5), then .
Proof. Since and , then Thus and a reduction is obtained in each iteration.
Lemma 5. If is the optimal solution of Problem (5), then
Proof. Let be the optimal solution of Problem (1); we can write . is a ball of center with radius . There are two cases:(i)If , then and Lemma 5 holds.(ii)If , since is convex, intersection point of the boundary of and the line segment between and should be feasible for Problem (5). Let be the intersection point; then satisfies for , and . Thus, Hence,and we have Thus,From (14) we also have ; therefore Substituting in the above inequality, we have . Since is convex and , then Furthermore, ; then .
Theorem 6. In every iteration of the algorithm MKKY, potential function is reduced by a constant value such that .
Proof. ConsiderWe used the result demonstrated by Karmarkar : Then , where .
If , then .
Theorem 7. If is the feasible solution of Problem (1) and is the optimal solution with optimal value , then one has following assumptions:(1)(2); for any feasible solution one has .
The algorithm MKKY finds an optimal solution after iteration. is the number of bytes.
Proof. Consider whereBy assumptions (1) and (2) we have in the feasible region; then According to Theorem 6, after iterations, we have . Thus, Therefore, for
In the next section MKKY algorithm is combined with the algorithm of Schrijver [6, 7] and Malek-Naseri’s algorithm  to propose novel hybrid algorithms called Sch-MKKY (Schrijver-Modified Kebiche-Keraghel-Yassine) and MN-MKKY (Malek-Naseri-Modified Kebiche-Keraghel-Yassine). These algorithms are different from MKKY in the use of optimal length in each iteration.
4.1. Hybrid Algorithms
Let us assume that and in the modified algorithm are expressed as follows: and . In the Sch-MKKY and MN-MKKY algorithms, choose and , respectively. It is easy to prove that the theorems in Section 4 satisfy Sch-MKKY and MN-MKKY algorithms. Thus with the recent step length the convergence is guaranteed.
5. Numerical Results
It is observed that the approximate solution from KKY algorithm is not feasible in some instances (see Examples 8, 9, and 11 in Table 1). All programs have been written with MATLAB 7.0.4 for as tolerance. The computed solution for KKY, Sch-MKKY, and MN-MKKY is given in Table 1.
Example 8. Consider the quadratic convex problem:
As it is shown, the KKY algorithm does not converge to the correct solution, while the computed solution of MKKY and two hybrid algorithms is feasible. Example 8 is solved by using Fmincon from MATLAB 7.0.4 and the absolute difference of the objective functions stated as Error. In Table 2 number of iterations, solution norms, optimal values of the objective function, Error, and the elapsed time for each algorithm have been given.
In Table 2 we show that MN-MKKY algorithm is more efficient than the other algorithms comparing the number of iterations and elapsed time.
In Figure 1, for various tolerances, the number of iterations is compared for different algorithms. It is observed that the MN-MKKY algorithm has better performance for each required tolerance.
(a) Example 8
(b) Example 9
(c) Example 10
(d) Example 11
Example 10. Consider the quadratic convex problem:
Example 11. Consider the nonlinear convex problem:
Having two ideas in our mind, (i) calculation of feasible solution in each iteration and (ii) the fact that the objective function value must decrease in each iteration with the fixed desired tolerance, this paper proposed three hybrid algorithms for solving nonlinear convex programming problem based on the interior point idea using various ’s of Karmarkar, Schrijver, and Malek-Naseri techniques. These methods have better performance than the standard Karmarkar algorithm, since in the latter algorithm one may not check the feasibility of solution in each iteration.
Our numerical simulation shows that the MN-MKKY algorithm has the best performance among the other algorithms. This algorithm uses less number of iterations to solve the general nonlinear optimization problems with linear constraints, since it uses the step length of Malek-Naseri type.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
M. S. Bazzara, J. Jarvis, and H. D. Sherali, Linear Programming and Network Flows, John Wiley & Sons, New York, NY, USA, 1984.
A. T. Hamdy, Operations Research, Macmillan, New York, NY, USA, 1992.View at: MathSciNet
C. Roos, T. Terlaky, and J. Vial, Theory and Algorithms for Linear Optimization, Princton University, 2001.
A. Schrijver, Theory of Linear and Integer Programming, John Wiley & Sons, New York, NY, USA, 1986.View at: MathSciNet