Iterative Methods and ApplicationsView this Special Issue
Research Article | Open Access
A Rank-Two Feasible Direction Algorithm for the Binary Quadratic Programming
Based on the semidefinite programming relaxation of the binary quadratic programming, a rank-two feasible direction algorithm is presented. The proposed algorithm restricts the rank of matrix variable to be two in the semidefinite programming relaxation and yields a quadratic objective function with simple quadratic constraints. A feasible direction algorithm is used to solve the nonlinear programming. The convergent analysis and time complexity of the method is given. Coupled with randomized algorithm, a suboptimal solution is obtained for the binary quadratic programming. At last, we report some numerical examples to compare our algorithm with randomized algorithm based on the interior point method and the feasible direction algorithm on max-cut problem. Simulation results have shown that our method is faster than the other two methods.
In this paper, we consider the following binary quadratic programming: where is real symmetric matrices and is a real -dimensional column vector.
The binary quadratic programming is a fundamental problem in optimization theory and practice. Some combinatorial optimization problems and engineering problems can be modeled as binary quadratic programming, such as VLSI design, statistical physics, max-cut problem , the optimal multiuser detection [2–5], image processing , and the design of FIR filters with discrete coefficients . These problems are known to be -hard . One typical approach to solve these problems is to construct lower bounds for approximating the optimal value. Now, the semidefinite programming (SDP) relaxation approach had been studied and proven to be quite powerful for finding approximate optimal solutions. Based on solving its semidefinite programming relaxation, Goemans and Williamson  developed a randomized algorithm for the max-cut problem, which provides an approximate solution guaranteed to be within a factor of 0.87856 of its optimal value. Interior point method is a powerful method for SDP with small and moderate scale. But the interior point method is limited to problems of moderate size, which cannot solve SDP with large scale efficiently . So Goemans and Williamson’s method based on the interior point method is not adapted to solve the large scale max-cut problems.
Some efficient nonlinear programming algorithms only based on gradient for solving the SDP relaxation of the max-cut problem have been developed. Homer and Peinado  proposed a parallel and distributed approximation algorithms for max-cut problem. In the algorithm, the author transformed the max-cut SDP relaxation into a constrained nonlinear programming problem in the new variable for using the change of variables , , where is the primal matrix variable of the SDP relaxation. Burer and Monteiro  proposed a projected gradient algorithm for solving the max-cut SDP relaxation by using the change of variable , where is a lower triangular matrix. The rank-two relaxation heuristics algorithm in  relaxed the max-cut problem to form an unconstrained optimization problem by replacing each binary variable with one unit vector in space and then using polar coordinates. In , the rank-two SDP relaxation model is proposed for maximal independent set problem. Based on the low-rank decomposition of the semidefinite matrix, Liu et al.  proposed a feasible direction method to solve a nonlinear programming model of binary quadratic programming.
In the paper, we propose a rank-two feasible direction method for the binary quadratic programming. we restrict the rank of matrix variable to be two in the semidefinite programming relaxation and obtain a quadratic objective function with simple quadratic constraints. A feasible direction method is used to solve the nonlinear programming. We also give the analysis of the convergence and the complexity of the method. The randomized algorithm is used to obtain the suboptimal solution of the binary quadratic programming. At last, we compare our method with the randomized algorithm based on the interior point method and the feasible direction method on max-cut problem. Simulation results show that our method costs less CPU time than the two methods.
2. The SDP Relaxation Method for Binary Quadratic Programming
In this section, we introduce the SDP relaxation of binary quadratic programming problem .
Let , where denotes the largest eigenvalue of the matrix and denotes the unit matrix; then is a negative definite matrix. Problem (2) is equivalent to the following problem below:
Letting and ignoring the constant term, then problem (3) is equivalent to the following problem: where and is the diagonal elements of matrix . In addition, denotes that matrix is semidefinite. Ignoring the nonconvex “rank one” constraint, the SDP relaxation is given as follows :
Interior point methods have been proved to be quite efficient for small and moderate scale SDP. In the 0.878 randomized method by Goemans and Williamson , the author solved the SDP relaxation problem (5) by interior point methods. However, interior point methods are second-order method, so they are quite time and memory intensive and not adapted to large scale binary quadratic problems. The complexity of the primal-dual interior point method based on AHO search direction for the SDP relaxation (5) of the max-cut is [14, 15].
3. The Rank-Two SDP Relaxation for Binary Quadratic Programming
In , the rank-two SDP relaxation model is proposed for max-cut problem based on the polar direction. In , the rank-two SDP relaxation model is proposed for maximal independent set problem. In this section, we present a rank-two SDP relaxation based on the rank-two approximate matrix for binary quadratic programming.
In SDP relaxation problem (5), let ; we have Obviously, matrix is positive definite.
Let , ; then and . We obtain the rank-two SDP relaxation of binary quadratic programming as follows: where and are the elements of vector and . Obviously, matrix satisfies that , , so problem (7) is a rank-two SDP relaxation problem.
Problem (7) is also a nonlinear programming with quadratic objective function and constraints. Compared to the variables of SDP relaxation, the rank-two relaxation has only variables, so this approach possesses scalability for solving large-scale binary quadratic programming problems.
Let then the gradients of the function and are
It is simple to obtain that Then we have the equivalent KKT condition for problem (7) as follows:
4. The Rank-Two Feasible Direction Algorithm for Binary Quadratic Programming
Feasible direction algorithm is an efficient algorithm for some special nonlinear programming problems. In , the feasible direction algorithm is applied to solve the low-rank nonlinear programming relaxation for binary quadratic programming problems. In , the feasible direction algorithm is applied to solve a rank one nonlinear programming relaxation for max-cut problem.
In this section, we extend the feasible direction algorithm to solve problem (7). The algorithm employs only gradient evaluations of the objective function in problem (7), and no calculations on any matrices and no line searches, thus greatly reduces the calculation costs and increases the efficiency of the algorithm.
In the rank-two feasible direction algorithm, we give the following iteration for problem (7): where denotes the element pair of matrix variable .
The iteration (13) is very simple and has the following characteristics.(1) No matrix calculations and no line searches are required, and only one gradient evaluation is needed to get the new iteration.(2) The new iteration point is feasible to problem (7).(3) If the sequence converges to , then is feasible to problem (7).
Define direction as follows: as a search direction, where is the elements pair of . Then the iteration (13) can be written as
Lemma 1. If , then is a point of (7).
The convergence of the feasible direction method is concluded by the following lemmas.
Lemma 3. Suppose that ; then any accumulation point is a point of (7).
Proof. Let be an accumulation point of the sequence ; it is simple to obtain the result by Lemma 1.
Lemma 4 (see ). Let matrixes and be positive definite; then is bounded by where and denote the smallest and the largest eigenvalues of the matrix .
Lemma 5. If for all , then .
Lemma 6. For all initial point and , the rank-two feasible direction algorithm terminates in iterations, where is an integer which does not exceed .
Proof. Based on Lemma 5, the number of iterations of the rank-two feasible direction algorithm is finite. Let be the number of iterations; then
So we obtain
Now we conclude that is an upper bound on the number of iterations.
Since problem (7) is nonconvex, there is no guarantee that the solution generated from the feasible direction method is a global solution. However, numerical experiments in Section 5 show that the proposed algorithm always converges to the optimal solution set of problem (7).
Now, we derive the complexity of our algorithm.
The complexity of computation of the gradient is . Each norm of the gradient of the objective function can be computed in , so we conclude that the overall complexity to evaluate the next iteration point is . Together with Lemma 6, we get that the overall complexity of the rank-two feasible direction algorithm is . Here we can choose satisfying ; then the complexity does not exceed . We know that the complexity of the primal-dual interior point method based on AHO direction is [14, 15]. It is obvious that the complexity of the primal-dual method is higher than that of our algorithm, so our algorithm is faster than the interior-point method for the large-scale SDP relaxation of binary quadratic programming problems. In addition, the complexity of low rank feasible direction method is , which is higher than that of our method.
Let the KKT point of problem (7) be ; then we can obtain the rank-two solution . Since the rank-two relaxation has the same form as Goemans and Williamson’s relaxation , except that ours has variables in rather than , the same analysis as Goeman and Williamson, with minimal changes, can be applied. By the randomized cut generation scheme, the suboptimal solution of binary quadratic programming problem is obtained.
5. Numerical Results
In this section we present computational results by comparing our method with GW randomized algorithm  based on interior point method and low-rank feasible direction algorithm to find approximate solutions to the max-cut problem.
In interior point method, we solve the SDP relaxation by three SDP solvers, which include SDPpack software , SeDuMi , and DSDP . SeDuMi is one of state-of-the-art SDP solvers. The code DSDP uses a dual-scaling interior-point algorithm and an iterative linear-equation solver. It is currently one of the fastest interior-point codes for solving SDP problems. Low-rank feasible direction algorithm is one of the efficient methods for the max-cut problem, and the algorithm is faster than the projected gradient algorithm . The projected gradient algorithm  is faster than Homer and Peinado algorithm .
All the algorithms are run in the MATLAB 7.0 environment on an Inter Core2 D2.0 GHz personal computer with 2.0 GB of RAM.
5.1. Max-Cut Problem
The max-cut problem is one of the standard -complete problems defined on graphs . Let denote an edge-weighted undirected graph without loops or multiple edges. We use , for an edge with endpoints and and for the weight of an edge . For the cut is the set of edges that have one endpoint in and the other endpoint in . The max-cut problem asks for the cut maximizing the sum of the weights of its edges. Here, we only work with the complete graph . In order to model a graph in this setting, define is referred to as the weighted adjacency matrix of the graph. An algebraic formulation can be obtained by introducing cut vectors . The max-cut problem can be formulated as the integer quadratic program as follows: The matrix is called the Laplace matrix of the graph , where is the unit vector whose every component is 1 and is the diagonal matrix whose diagonal elements are . Let ; the max-cut problem may be interpreted as a special case of the problem (1).
5.2. Numerical Results for the Random Graphs
The first set of test problems contains random graphs with two different edge densities 0.8 and 0.2, which denotes the dense random graphs and sparse random graphs, respectively. The weight on each edge is 1. We select problems in size from to for comparing the suboptimal value of max-cut problem and the CPU time of the four methods.
For the interior point method, we use the codes by two SDP solvers, which include SDPpack software  and SeDuMi . In our algorithm, the iteration stops once is found. The result is shown in Table 1.
In Table 1, “SDPpack” stands for randomized algorithm based on interior point method solved by SDPpack software, “SeDuMi” for randomized algorithm based on interior point method solved by SeDuMi software, “FD” for feasible direction algorithm coupled with the randomized method, “R2FD” for our rank-two feasible direction algorithm coupled with the randomized method, “CPU” for the CPU time, “Values” for the suboptimal value of the max-cut problem based these methods, and “Density” for edge density of the random graphs.
The SDPpack and SeDuMi provide the currently best conclusion on its performance guarantee in theory. The results in Table 1 show that the approximate solutions obtained by R2FD are as good as those generated by SDPpack, SeDuMi, and FD. In addition, the CPU time of our method is less than that of SDPpack, SeDuMi, and FD. In particular, with the increase of the size of the max-cut problem, the ratios of the CPU time between our methods to the three methods decrease quickly.
5.3. Numerical Results for the G-Set Graphs
The second set of test problems are from the so-called G-set graphs, which are randomly generated by the procedure rudy, a machine independent graph generator written by Rinaldi , Helmberg and Rendl , and Alperin and Nowak . [20–22]. The test problems include 14 randomly generated large size test problems with nodes from 800 to 2000. Recently, Choi and Ye  reported computational results on a subset of G-set graphs that were solved as max-cut problems using their SDP code COPL-DSDP, or simply DSDP. The code DSDP uses a dual-scaling interior-point algorithm and an iterative linear-equation solver. The SDPpack software does not work when the size of the max-cut problems is larger than 350, so we give the results by the randomized method based on the dual-scaling algorithm solved by the DSDP software .
Table 2 gives the results of comparison among our R2FD method, the FD method, and the randomized method based on DSDP and SeDuMi on 14 large size test problems in the second set. In Table 2, “DSDP” presents the randomized method based on the dual-scaling algorithm by the DSDP software.
The results in Table 2 show that the approximate solutions by our method is nearly as good as those of the DSDP cuts. But our method which reaches solutions of the problems is at least 10 times faster than the FD method, 7 times faster than DSDP, and 100 times faster than SeDuMi. In particular, for G35, G36, and G37, the CPU time of our method is almost 40 times less than that of DSDP. Furthermore, We observe that R2FD took less than 5 minutes to return approximate solutions to all the 14 test problems, which required more than 2 hours of computation by the DSDP, more than 11 hours of computation by the FD, and more than 22 hours of computation by the SeDuMi.
Because the interior-point method and feasible direction method increase the dimension of a problem from to and ( is the function of ), so the two methods cost more CPU time than our method for solving large size binary quadratic programming problems, especially for problems with a large number of edges. The rank-two feasible direction method only increases the dimension of a problem from to , so it is efficient for solving large size binary quadratic programming.
The work of Xuewen Mu is supported by the National Science Foundation for Young Scientists of China (Grant nos. 11101320 and 61201297) and the Fundamental Research Funds for the Central Universities (Grant no. K50511700007). The work of Yaling Zhang is supported by the Xian University of Science and Technology Cultivation Foundation in Shaan Xi Province of China (Program no. 2010032).
- C. Helmberg, Semidefinite Programming for Combinatorial Optimization, Konrad-Zuse-Zentrum fur Information-Stechnik, Berlin, Germany, 2000.
- F. Barahona, M. Groetschel, M. Juenger, and G. Reinelt, “An application of combinatorial optimization to statiscal optimization and circuit layout design,” Operations Research, vol. 36, no. 3, pp. 493–513, 1988.
- P. H. Tan and L. K. Rasmussen, “The application of semidefinite programming for detection in CDMA,” IEEE Journal on Selected Areas in Communications, vol. 19, no. 8, pp. 1442–1449, 2001.
- F. Hasegawa, J. Luo, K. Pattipati, and P. Willett, “Speed and accuracy comparation of techniques to solve a binary programming problem with application to syschronous CDMA,” IEEE Transaction on Communication, vol. 52, pp. 2775–2780, 2004.
- X. Mu and Y. Zhang, “A new rank-two semidefinite programming relaxation method for multiuser detection problem,” Wireless Personal Communications, vol. 65, pp. 223–233, 2012.
- J. Keuchel, C. Schnörr, C. Schellewald, and D. Cremers, “Binary partitioning, perceptual grouping, and restoration with semidefinite programming,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp. 1364–1379, 2003.
- S.-P. Wu, S. Boyd, and L. Vandenberghe, “FIR filter design via semidefinite programming and spectral factorization,” in Proceedings of the 35th IEEE Conference on Decision and Control, pp. 271–276, Kobe, Japan, December 1996.
- M. X. Goemans and D. P. Williamson, “Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming,” Journal of the Association for Computing Machinery, vol. 42, no. 6, pp. 1115–1145, 1995.
- S. Homer and M. Peinado, “Design and performance of parallel and distributed approximation algorithms for maxcut,” Journal of Parallel and Distributed Computing, vol. 46, no. 1, pp. 48–61, 1997.
- S. Burer and R. D. C. Monteiro, “A projected gradient algorithm for solving the maxcut SDP relaxation,” Optimization Methods and Software, vol. 15, no. 3-4, pp. 175–200, 2001.
- S. Burer, R. D. C. Monteiro, and Y. Zhang, “Rank-two relaxation heuristics for max-cut and other binary quadratic programs,” SIAM Journal on Optimization, vol. 12, no. 2, pp. 503–521, 2001.
- S. Burer, R. D. C. Monteiro, and Y. Zhang, “Maximum stable set formulations and heuristics based on continuous optimization,” Mathematical Programming A, vol. 94, no. 1, pp. 137–166, 2002.
- H. Liu, X. Wang, and S. Liu, “Feasible direction algorithm for solving the SDP relaxations of quadratic programming problems,” Optimization Methods & Software, vol. 19, no. 2, pp. 125–136, 2004.
- F. Alizadeh, P. A. Haeberly, and M. L. Overton, “Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results,” SIAM Journal on Optimization, vol. 8, no. 3, pp. 746–768, 1998.
- M. J. Todd, “Semidefinite optimization,” Acta Numerica, vol. 10, pp. 515–560, 2001.
- C.-x. Xu, X.-l. He, and F.-m. Xu, “An effective continuous algorithm for approximate solutions of large scale max-cut problems,” Journal of Computational Mathematics, vol. 24, no. 6, pp. 749–760, 2006.
- F. Alizadeh, J. P. Haeberly, M. V. Nayakkankuppam, M. L. Overton, and S. Schmieta, “SDPpack user’s guide-version 0.9Beta,” Tech. Rep. TR1997-737, Courant Institute of Mathematical Science, New York, NY, USA, June 1997.
- J. F. Sturm, “Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones,” Optimization Methods and Software, vol. 11-12, no. 1–4, pp. 625–653, 1999.
- C. Choi and Y. Ye, “Solving Sparse Semidefinite Programs Using the Dual Scaling Algorithm with an Iterative Solver,” Working Paper, Department of Management Science, University of Iowa, Iowa City, Iowa, USA, 2000.
- G. Rinaldi, “Rudy graph generator,” http://www-user.tu-chemnitz.de/ helmberg/rudy.tar.gz.
- C. Helmberg and F. Rendl, “A spectral bundle method for semidefinite programming,” SIAM Journal on Optimization, vol. 10, no. 3, pp. 673–696, 2000.
- H. Alperin and I. Nowak, “Lagrangian smoothing heuristics for max-cut,” Journal of Heuristics, vol. 11, no. 5-6, pp. 447–463, 2005.
Copyright © 2013 Xuewen Mu and Yaling Zhang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.