Research Article  Open Access
Xuewen Mu, Yaling Zhang, "A RankTwo Feasible Direction Algorithm for the Binary Quadratic Programming", Journal of Applied Mathematics, vol. 2013, Article ID 963563, 7 pages, 2013. https://doi.org/10.1155/2013/963563
A RankTwo Feasible Direction Algorithm for the Binary Quadratic Programming
Abstract
Based on the semidefinite programming relaxation of the binary quadratic programming, a ranktwo feasible direction algorithm is presented. The proposed algorithm restricts the rank of matrix variable to be two in the semidefinite programming relaxation and yields a quadratic objective function with simple quadratic constraints. A feasible direction algorithm is used to solve the nonlinear programming. The convergent analysis and time complexity of the method is given. Coupled with randomized algorithm, a suboptimal solution is obtained for the binary quadratic programming. At last, we report some numerical examples to compare our algorithm with randomized algorithm based on the interior point method and the feasible direction algorithm on maxcut problem. Simulation results have shown that our method is faster than the other two methods.
1. Introduction
In this paper, we consider the following binary quadratic programming: where is real symmetric matrices and is a real dimensional column vector.
The binary quadratic programming is a fundamental problem in optimization theory and practice. Some combinatorial optimization problems and engineering problems can be modeled as binary quadratic programming, such as VLSI design, statistical physics, maxcut problem [1], the optimal multiuser detection [2–5], image processing [6], and the design of FIR filters with discrete coefficients [7]. These problems are known to be hard [1]. One typical approach to solve these problems is to construct lower bounds for approximating the optimal value. Now, the semidefinite programming (SDP) relaxation approach had been studied and proven to be quite powerful for finding approximate optimal solutions. Based on solving its semidefinite programming relaxation, Goemans and Williamson [8] developed a randomized algorithm for the maxcut problem, which provides an approximate solution guaranteed to be within a factor of 0.87856 of its optimal value. Interior point method is a powerful method for SDP with small and moderate scale. But the interior point method is limited to problems of moderate size, which cannot solve SDP with large scale efficiently [1]. So Goemans and Williamson’s method based on the interior point method is not adapted to solve the large scale maxcut problems.
Some efficient nonlinear programming algorithms only based on gradient for solving the SDP relaxation of the maxcut problem have been developed. Homer and Peinado [9] proposed a parallel and distributed approximation algorithms for maxcut problem. In the algorithm, the author transformed the maxcut SDP relaxation into a constrained nonlinear programming problem in the new variable for using the change of variables , , where is the primal matrix variable of the SDP relaxation. Burer and Monteiro [10] proposed a projected gradient algorithm for solving the maxcut SDP relaxation by using the change of variable , where is a lower triangular matrix. The ranktwo relaxation heuristics algorithm in [11] relaxed the maxcut problem to form an unconstrained optimization problem by replacing each binary variable with one unit vector in space and then using polar coordinates. In [12], the ranktwo SDP relaxation model is proposed for maximal independent set problem. Based on the lowrank decomposition of the semidefinite matrix, Liu et al. [13] proposed a feasible direction method to solve a nonlinear programming model of binary quadratic programming.
In the paper, we propose a ranktwo feasible direction method for the binary quadratic programming. we restrict the rank of matrix variable to be two in the semidefinite programming relaxation and obtain a quadratic objective function with simple quadratic constraints. A feasible direction method is used to solve the nonlinear programming. We also give the analysis of the convergence and the complexity of the method. The randomized algorithm is used to obtain the suboptimal solution of the binary quadratic programming. At last, we compare our method with the randomized algorithm based on the interior point method and the feasible direction method on maxcut problem. Simulation results show that our method costs less CPU time than the two methods.
2. The SDP Relaxation Method for Binary Quadratic Programming
In this section, we introduce the SDP relaxation of binary quadratic programming problem [1].
In problem (1), let , , and ; then problem (1) can be formulated as
It is well known that problem (2) is also hard [1].
Let , where denotes the largest eigenvalue of the matrix and denotes the unit matrix; then is a negative definite matrix. Problem (2) is equivalent to the following problem below:
Letting and ignoring the constant term, then problem (3) is equivalent to the following problem: where and is the diagonal elements of matrix . In addition, denotes that matrix is semidefinite. Ignoring the nonconvex “rank one” constraint, the SDP relaxation is given as follows [1]:
Interior point methods have been proved to be quite efficient for small and moderate scale SDP. In the 0.878 randomized method by Goemans and Williamson [8], the author solved the SDP relaxation problem (5) by interior point methods. However, interior point methods are secondorder method, so they are quite time and memory intensive and not adapted to large scale binary quadratic problems. The complexity of the primaldual interior point method based on AHO search direction for the SDP relaxation (5) of the maxcut is [14, 15].
3. The RankTwo SDP Relaxation for Binary Quadratic Programming
In [11], the ranktwo SDP relaxation model is proposed for maxcut problem based on the polar direction. In [12], the ranktwo SDP relaxation model is proposed for maximal independent set problem. In this section, we present a ranktwo SDP relaxation based on the ranktwo approximate matrix for binary quadratic programming.
In SDP relaxation problem (5), let ; we have Obviously, matrix is positive definite.
Let , [12]; then and . We obtain the ranktwo SDP relaxation of binary quadratic programming as follows: where and are the elements of vector and . Obviously, matrix satisfies that , , so problem (7) is a ranktwo SDP relaxation problem.
Problem (7) is also a nonlinear programming with quadratic objective function and constraints. Compared to the variables of SDP relaxation, the ranktwo relaxation has only variables, so this approach possesses scalability for solving largescale binary quadratic programming problems.
Let then the gradients of the function and are
The KKT condition for problem (7) is given here. If the variable in problem (7) satisfies the following condition: then is a KKT point for problem (7).
It is simple to obtain that Then we have the equivalent KKT condition for problem (7) as follows:
4. The RankTwo Feasible Direction Algorithm for Binary Quadratic Programming
Feasible direction algorithm is an efficient algorithm for some special nonlinear programming problems. In [13], the feasible direction algorithm is applied to solve the lowrank nonlinear programming relaxation for binary quadratic programming problems. In [16], the feasible direction algorithm is applied to solve a rank one nonlinear programming relaxation for maxcut problem.
In this section, we extend the feasible direction algorithm to solve problem (7). The algorithm employs only gradient evaluations of the objective function in problem (7), and no calculations on any matrices and no line searches, thus greatly reduces the calculation costs and increases the efficiency of the algorithm.
In the ranktwo feasible direction algorithm, we give the following iteration for problem (7): where denotes the element pair of matrix variable .
The iteration (13) is very simple and has the following characteristics.(1) No matrix calculations and no line searches are required, and only one gradient evaluation is needed to get the new iteration.(2) The new iteration point is feasible to problem (7).(3) If the sequence converges to , then is feasible to problem (7).
Define direction as follows: as a search direction, where is the elements pair of . Then the iteration (13) can be written as
The following lemmas show that if , then is a KKT point of problem (7), and if , then is a feasible increasing direction for problem (7).
Lemma 1. If , then is a point of (7).
Proof. It is clear that satisfies the constraint in problem (7). Since , then Let ; by the KKT condition (12), we have that is a KKT point of (7). This completes the proof of the lemma.
Lemma 2. Suppose that ; then is a feasible increasing direction for problem (7), and iteration point is feasible to problem (7).
Proof. The feasibility of the iteration point directly comes from definition (13). Using the fact that , we have So direction is a feasible increasing direction for problem (7).
The convergence of the feasible direction method is concluded by the following lemmas.
Lemma 3. Suppose that ; then any accumulation point is a point of (7).
Proof. Let be an accumulation point of the sequence ; it is simple to obtain the result by Lemma 1.
Lemma 4 (see [1]). Let matrixes and be positive definite; then is bounded by where and denote the smallest and the largest eigenvalues of the matrix .
Lemma 5. If for all , then .
Proof. Since
from Lemma 4, we have
where denotes the Frobenius norm matrix .
From Lemma 2 and Lemma 4, for any , we have
This shows that is convergent, and hence holds.
In view of Lemmas 1 and 5, the termination criterion used in the ranktwo feasible algorithm is , where is a prespecified constant.
Lemma 6. For all initial point and , the ranktwo feasible direction algorithm terminates in iterations, where is an integer which does not exceed .
Proof. Based on Lemma 5, the number of iterations of the ranktwo feasible direction algorithm is finite. Let be the number of iterations; then
So we obtain
Now we conclude that is an upper bound on the number of iterations.
Since problem (7) is nonconvex, there is no guarantee that the solution generated from the feasible direction method is a global solution. However, numerical experiments in Section 5 show that the proposed algorithm always converges to the optimal solution set of problem (7).
Now, we derive the complexity of our algorithm.
The complexity of computation of the gradient is . Each norm of the gradient of the objective function can be computed in , so we conclude that the overall complexity to evaluate the next iteration point is . Together with Lemma 6, we get that the overall complexity of the ranktwo feasible direction algorithm is . Here we can choose satisfying ; then the complexity does not exceed . We know that the complexity of the primaldual interior point method based on AHO direction is [14, 15]. It is obvious that the complexity of the primaldual method is higher than that of our algorithm, so our algorithm is faster than the interiorpoint method for the largescale SDP relaxation of binary quadratic programming problems. In addition, the complexity of low rank feasible direction method is [13], which is higher than that of our method.
Let the KKT point of problem (7) be ; then we can obtain the ranktwo solution . Since the ranktwo relaxation has the same form as Goemans and Williamson’s relaxation [8], except that ours has variables in rather than , the same analysis as Goeman and Williamson, with minimal changes, can be applied. By the randomized cut generation scheme, the suboptimal solution of binary quadratic programming problem is obtained.
5. Numerical Results
In this section we present computational results by comparing our method with GW randomized algorithm [8] based on interior point method and lowrank feasible direction algorithm to find approximate solutions to the maxcut problem.
In interior point method, we solve the SDP relaxation by three SDP solvers, which include SDPpack software [17], SeDuMi [18], and DSDP [19]. SeDuMi is one of stateoftheart SDP solvers. The code DSDP uses a dualscaling interiorpoint algorithm and an iterative linearequation solver. It is currently one of the fastest interiorpoint codes for solving SDP problems. Lowrank feasible direction algorithm is one of the efficient methods for the maxcut problem, and the algorithm is faster than the projected gradient algorithm [13]. The projected gradient algorithm [10] is faster than Homer and Peinado algorithm [9].
All the algorithms are run in the MATLAB 7.0 environment on an Inter Core2 D2.0 GHz personal computer with 2.0 GB of RAM.
5.1. MaxCut Problem
The maxcut problem is one of the standard complete problems defined on graphs [8]. Let denote an edgeweighted undirected graph without loops or multiple edges. We use , for an edge with endpoints and and for the weight of an edge . For the cut is the set of edges that have one endpoint in and the other endpoint in . The maxcut problem asks for the cut maximizing the sum of the weights of its edges. Here, we only work with the complete graph . In order to model a graph in this setting, define is referred to as the weighted adjacency matrix of the graph. An algebraic formulation can be obtained by introducing cut vectors . The maxcut problem can be formulated as the integer quadratic program as follows: The matrix is called the Laplace matrix of the graph , where is the unit vector whose every component is 1 and is the diagonal matrix whose diagonal elements are . Let ; the maxcut problem may be interpreted as a special case of the problem (1).
5.2. Numerical Results for the Random Graphs
The first set of test problems contains random graphs with two different edge densities 0.8 and 0.2, which denotes the dense random graphs and sparse random graphs, respectively. The weight on each edge is 1. We select problems in size from to for comparing the suboptimal value of maxcut problem and the CPU time of the four methods.
For the interior point method, we use the codes by two SDP solvers, which include SDPpack software [17] and SeDuMi [18]. In our algorithm, the iteration stops once is found. The result is shown in Table 1.

In Table 1, “SDPpack” stands for randomized algorithm based on interior point method solved by SDPpack software, “SeDuMi” for randomized algorithm based on interior point method solved by SeDuMi software, “FD” for feasible direction algorithm coupled with the randomized method, “R2FD” for our ranktwo feasible direction algorithm coupled with the randomized method, “CPU” for the CPU time, “Values” for the suboptimal value of the maxcut problem based these methods, and “Density” for edge density of the random graphs.
The SDPpack and SeDuMi provide the currently best conclusion on its performance guarantee in theory. The results in Table 1 show that the approximate solutions obtained by R2FD are as good as those generated by SDPpack, SeDuMi, and FD. In addition, the CPU time of our method is less than that of SDPpack, SeDuMi, and FD. In particular, with the increase of the size of the maxcut problem, the ratios of the CPU time between our methods to the three methods decrease quickly.
5.3. Numerical Results for the GSet Graphs
The second set of test problems are from the socalled Gset graphs, which are randomly generated by the procedure rudy, a machine independent graph generator written by Rinaldi [20], Helmberg and Rendl [21], and Alperin and Nowak [22]. [20–22]. The test problems include 14 randomly generated large size test problems with nodes from 800 to 2000. Recently, Choi and Ye [19] reported computational results on a subset of Gset graphs that were solved as maxcut problems using their SDP code COPLDSDP, or simply DSDP. The code DSDP uses a dualscaling interiorpoint algorithm and an iterative linearequation solver. The SDPpack software does not work when the size of the maxcut problems is larger than 350, so we give the results by the randomized method based on the dualscaling algorithm solved by the DSDP software [19].
Table 2 gives the results of comparison among our R2FD method, the FD method, and the randomized method based on DSDP and SeDuMi on 14 large size test problems in the second set. In Table 2, “DSDP” presents the randomized method based on the dualscaling algorithm by the DSDP software.

The results in Table 2 show that the approximate solutions by our method is nearly as good as those of the DSDP cuts. But our method which reaches solutions of the problems is at least 10 times faster than the FD method, 7 times faster than DSDP, and 100 times faster than SeDuMi. In particular, for G35, G36, and G37, the CPU time of our method is almost 40 times less than that of DSDP. Furthermore, We observe that R2FD took less than 5 minutes to return approximate solutions to all the 14 test problems, which required more than 2 hours of computation by the DSDP, more than 11 hours of computation by the FD, and more than 22 hours of computation by the SeDuMi.
6. Conclusion
Because the interiorpoint method and feasible direction method increase the dimension of a problem from to and ( is the function of ), so the two methods cost more CPU time than our method for solving large size binary quadratic programming problems, especially for problems with a large number of edges. The ranktwo feasible direction method only increases the dimension of a problem from to , so it is efficient for solving large size binary quadratic programming.
Acknowledgments
The work of Xuewen Mu is supported by the National Science Foundation for Young Scientists of China (Grant nos. 11101320 and 61201297) and the Fundamental Research Funds for the Central Universities (Grant no. K50511700007). The work of Yaling Zhang is supported by the Xian University of Science and Technology Cultivation Foundation in Shaan Xi Province of China (Program no. 2010032).
References
 C. Helmberg, Semidefinite Programming for Combinatorial Optimization, KonradZuseZentrum fur InformationStechnik, Berlin, Germany, 2000.
 F. Barahona, M. Groetschel, M. Juenger, and G. Reinelt, “An application of combinatorial optimization to statiscal optimization and circuit layout design,” Operations Research, vol. 36, no. 3, pp. 493–513, 1988. View at: Publisher Site  Google Scholar  Zentralblatt MATH
 P. H. Tan and L. K. Rasmussen, “The application of semidefinite programming for detection in CDMA,” IEEE Journal on Selected Areas in Communications, vol. 19, no. 8, pp. 1442–1449, 2001. View at: Publisher Site  Google Scholar
 F. Hasegawa, J. Luo, K. Pattipati, and P. Willett, “Speed and accuracy comparation of techniques to solve a binary programming problem with application to syschronous CDMA,” IEEE Transaction on Communication, vol. 52, pp. 2775–2780, 2004. View at: Google Scholar
 X. Mu and Y. Zhang, “A new ranktwo semidefinite programming relaxation method for multiuser detection problem,” Wireless Personal Communications, vol. 65, pp. 223–233, 2012. View at: Publisher Site  Google Scholar
 J. Keuchel, C. Schnörr, C. Schellewald, and D. Cremers, “Binary partitioning, perceptual grouping, and restoration with semidefinite programming,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp. 1364–1379, 2003. View at: Publisher Site  Google Scholar
 S.P. Wu, S. Boyd, and L. Vandenberghe, “FIR filter design via semidefinite programming and spectral factorization,” in Proceedings of the 35th IEEE Conference on Decision and Control, pp. 271–276, Kobe, Japan, December 1996. View at: Publisher Site  Google Scholar
 M. X. Goemans and D. P. Williamson, “Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming,” Journal of the Association for Computing Machinery, vol. 42, no. 6, pp. 1115–1145, 1995. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 S. Homer and M. Peinado, “Design and performance of parallel and distributed approximation algorithms for maxcut,” Journal of Parallel and Distributed Computing, vol. 46, no. 1, pp. 48–61, 1997. View at: Publisher Site  Google Scholar
 S. Burer and R. D. C. Monteiro, “A projected gradient algorithm for solving the maxcut SDP relaxation,” Optimization Methods and Software, vol. 15, no. 34, pp. 175–200, 2001. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 S. Burer, R. D. C. Monteiro, and Y. Zhang, “Ranktwo relaxation heuristics for maxcut and other binary quadratic programs,” SIAM Journal on Optimization, vol. 12, no. 2, pp. 503–521, 2001. View at: Publisher Site  Google Scholar  MathSciNet
 S. Burer, R. D. C. Monteiro, and Y. Zhang, “Maximum stable set formulations and heuristics based on continuous optimization,” Mathematical Programming A, vol. 94, no. 1, pp. 137–166, 2002. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H. Liu, X. Wang, and S. Liu, “Feasible direction algorithm for solving the SDP relaxations of quadratic $\{1,1\}$ programming problems,” Optimization Methods & Software, vol. 19, no. 2, pp. 125–136, 2004. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Alizadeh, P. A. Haeberly, and M. L. Overton, “Primaldual interiorpoint methods for semidefinite programming: convergence rates, stability and numerical results,” SIAM Journal on Optimization, vol. 8, no. 3, pp. 746–768, 1998. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 M. J. Todd, “Semidefinite optimization,” Acta Numerica, vol. 10, pp. 515–560, 2001. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 C.x. Xu, X.l. He, and F.m. Xu, “An effective continuous algorithm for approximate solutions of large scale maxcut problems,” Journal of Computational Mathematics, vol. 24, no. 6, pp. 749–760, 2006. View at: Google Scholar  Zentralblatt MATH  MathSciNet
 F. Alizadeh, J. P. Haeberly, M. V. Nayakkankuppam, M. L. Overton, and S. Schmieta, “SDPpack user’s guideversion 0.9Beta,” Tech. Rep. TR1997737, Courant Institute of Mathematical Science, New York, NY, USA, June 1997. View at: Google Scholar
 J. F. Sturm, “Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones,” Optimization Methods and Software, vol. 1112, no. 1–4, pp. 625–653, 1999. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 C. Choi and Y. Ye, “Solving Sparse Semidefinite Programs Using the Dual Scaling Algorithm with an Iterative Solver,” Working Paper, Department of Management Science, University of Iowa, Iowa City, Iowa, USA, 2000. View at: Google Scholar
 G. Rinaldi, “Rudy graph generator,” http://wwwuser.tuchemnitz.de/ helmberg/rudy.tar.gz. View at: Google Scholar
 C. Helmberg and F. Rendl, “A spectral bundle method for semidefinite programming,” SIAM Journal on Optimization, vol. 10, no. 3, pp. 673–696, 2000. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H. Alperin and I. Nowak, “Lagrangian smoothing heuristics for maxcut,” Journal of Heuristics, vol. 11, no. 56, pp. 447–463, 2005. View at: Publisher Site  Google Scholar  Zentralblatt MATH
Copyright
Copyright © 2013 Xuewen Mu and Yaling Zhang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.