A Branch-and-Bound Algorithm Embedded with DCA for DC Programming
The special importance of Difference of Convex (DC) functions programming has been recognized in recent studies on nonconvex optimization problems. In this work, a class of DC programming derived from the portfolio selection problems is studied. The most popular method applied to solve the problem is the Branch-and-Bound (B&B) algorithm. However, “the curse of dimensionality” will affect the performance of the B&B algorithm. DC Algorithm (DCA) is an efficient method to get a local optimal solution. It has been applied to many practical problems, especially for large-scale problems. A B&B-DCA algorithm is proposed by embedding DCA into the B&B algorithms, the new algorithm improves the computational performance and obtains a global optimal solution. Computational results show that the proposed B&B-DCA algorithm has the superiority of the branch number and computational time than general B&B. The nice features of DCA (inexpensiveness, reliability, robustness, globality of computed solutions, etc.) provide crucial support to the combined B&B-DCA for accelerating the convergence of B&B.
DC programming is an important subject in optimization problems. This paper studies one class of DC programming, which is originally derived from the portfolio investment problems.
Consider the following problem: here , with being a positive definite matrix. , with being a concave function for . Decision vector , with being a polyhedron, . In addition, is restricted to the lower bounds and the upper bounds .
Falk and Soland propose a B&B algorithm for separable nonconvex programming problems in , where the objective function is a separable nonconvex function. Phong et al. give a decomposition B&B method for globally solving linearly constrained indefinite quadratic minimization problems in [2, 3], where the objective function is with being a convex quadratic function, however, the concave part is a function of rather than . Konno and Wijayanayake  propose a B&B algorithm to solve portfolio optimization problems under concave transaction costs. The algorithm is proposed by introducing linear underestimated functions for concave transaction cost functions, and is successively used to solve optimal portfolio selection problems with 200 assets. Honggang and Chengxian give a B&B algorithm to solve this class of DC programming with the proposed largest distance bisection in , and tests show the efficiency of the method for the problem with 20–160 dimensions. More representatively, a convex minorant of the DC function is defined by for B&B algorithm in DC programming, where denotes the convex envelope of the concave function on the set , and this is called DC relaxation in DC programming which has been completely studied in  and is important for nonconvex programming problems. The performance of the B&B depends on the branching strategy and bounding technology. The main concern of the above B&B algorithms is to solve the underestimated problem of initial problem to obtain an upper bound and also a lower bound for the optimal value, then divide the problem into two subproblems according to some rules, and repeat the above steps for a selected subproblem. By constantly reducing the upper bound and increasing the lower bound for the optimal value, we can obtain a global optimal solution. The main contribution of our work is to improve the upper bound for the optimal value by a local optimization algorithm for the DC programming self rather than the underestimated problem.
DCA is an effective local optimization method based on local optimality and the duality for solving DC programming, especially for large-scale problems. DCA has been first proposed by Tao and An  in their preliminary form in 1985 as an extension of subgradient algorithms to DC programming. Then, the method becomes classic and increasingly popular due to the joint work by An and Tao since 1994. Crucial developments and improvements for DCA from both theoretical and computational aspects have been completed, see [7–12] and references therein. In particular, the work by An et al.  investigates DC Programming and DCA for solving large-scale (until 400 000 dimensions) nonconvex quadratic programming. Since solving large-scale convex quadratic programs is expensive, the combination of DCA and interior point method (IP) can reduce the computation time and this outperforms the reference code LOQO by Vanderbei (Princeton). DCA has been widely used to a lot of different and nonconvex optimization problems, such as trust region subproblem, nonconvex quadratic programming, and so on (see [6, 13–17]). Furthermore, it always provides a global optimal solution for the problems . The very complete reference including a list of real-world applications of DC Programming and DCA has been summarized by An and Tao(see the website ).
DCA is an efficient method for DC programming which allows to solve large-scale DC programming. In this paper, we will obtain a local optimization solution by the DCA method, the optimal value of which is also an upper bound for the optimal value of the problem (). In most cases, the local optimal solution is always a global optimal solution. Embedding this upper bound into the B&B algorithms can improve the convergence speed and guarantee the global optimality of the solution when it is used to solve the problem (). Computational tests will be conducted for general problem and especially for portfolio selection problem, the results show that the proposed B&B-DCA algorithm can solve problems efficiently and have the superiority of the branch number and computational time.
The rest of the paper is organized as follows. Local optimization method DCA for the problem () is described in Section 2. The B&B method embedded with DCA algorithm is given in Section 3. Computational tests are stated in Section 4. Conclusion and future research are shown in Section 5.
2. Local Optimization Method DCA
2.1. DCA for General DC Programming
Consider the following general DC programming: where and are lower semicontinuous proper convex functions on . Such a function is called a DC function, and is called a decomposition of , while and are DC components of . In addition, a constrained DC programming whose feasible solution set is convex can also be transformed into an unconstrained DC programming by adding the indicator function of (it is equal to 0 in , infinity elsewhere) to the first DC component .
Let be the conjugate function of . Then, the dual programming of () can be expressed as The perfect symmetry exists between primal programming () and dual programming (): the dual of () is exactly (). Remark that if the optimal value is finite, we have here, . Such inclusion will be assumed throughout the paper.
Let and denote global solutions sets of problem () and (), respectively. According to the work by Toland , the relationship between them is Under technical conditions, this transportation also holds true for local optimal solutions of problems () and (), more details can be found in [8–11].
Based on local optimality conditions and duality in DC programming, the DCA consists in the construction of two sequence and , such that the sequences and are decreasing, and (resp., ) converges to a primal feasible solution (resp., a dual feasible solution ) which satisfies local optimality conditions and
In the following, we will show main convergence properties of DCA which have been proposed and proven in [10, 11, 17]. First, (resp., ) is used to denote a convex set containing the sequence (resp., ) and (or if ) the modulus of strong convexity of the function on are given by:(1)The sequences and are decreasing and.(i) if , and . Furthermore, if or are strictly convex on , .(ii) if , and . Furthermore, if or are strictly convex on then . In such cases, DCA terminates at the kth iteration (finite convergence of DCA).(2)If (resp., ), then the series (resp., ) converges.(3)If the optimal value of the primal problem () if finite and the infinite sequence and are bounded, then every limit point (resp., ) of the sequence (resp., ) is a critical point of (resp., ).(4)DCA has a linear convergence rate for general DC programming.
Problem () is a special form of general DC programming (), with and . According to the description of DCA in Section 2.1, we need to compute and . According to knowledge of modern convex analysis, we have .
As can be seen, if and only if is differential at , reduces to a singleton which is exactly . For the computation of , we need to solve the following convex quadratic programming: because its solution is exactly . Finally, DCA applied for solving problem () can be described as follows.
Algorithm 2.1 (The DCA for the Problem ()).
Let be a sufficiently small positive number. First select an initial point . Set , goto 2°. 2° Iteration
Set , that is, , and then solve the following quadratic programming the solution of (2.10) is denoted as , goto 3°. 3° Stop Criterion
If , then stop and we get a local optimal solution . Otherwise, set , goto 2°.
We can obtain a local optimal solution by Algorithm 2.1 efficiently for the problem () with different dimensions.
3. B&B Algorithm Embedded with DCA Methods
In most cases, B&B  methods are used to obtain the global optimal solution of the problem (). The main concern of the existing B&B algorithms is to solve the underestimated problem of the problem (). However, the computational cost of the algorithm will be very large along with the increasing dimension of the problem. In this section, we improve the upper bound for the optimal value by the local optimization algorithm DCA for DC programming self rather than the underestimated problem. DCA will be embedded into the B&B algorithm to accelerate the convergence of B&B.
3.1. The Description of the B&B-DCA Algorithm
In this subsection, we present the B&B-DCA method for the problem (). Let be the initial set which needs to be branched. We replace each concave function in by its underestimated function over the set : where Then, we let be the underestimated function of the objective in problem (). We solve the following quadratic programming problem:
Then, Algorithm 2.1 is used to obtain a local optimal solution for which is set as the initial iteration point . The relative optimal solution when Algorithm 2.1 stops are noted as . Then, we set the upper bound for the optimal value (where is a global optimal solution of the problem ()) .
Proof. The following relationship holds true: This gives the conclusion.
Before continuing to describe the algorithm, we need to know the “Rectangle Subdivision Process”, that is, divide the set into a sequence of subsets by means of hyperplanes parallel to certain facets . The family of subrectangles can be represented by a tree with root and subnodes. A node is a successor of another one if and only if it represents an element of the latter node. An infinite path in the tree corresponds to an infinite nested sequence of rectangles , . “Rectangle Subdivision Process” plays an important role in the B&B method. In order to ensure the convergence of the algorithm, the concept of “Normal Rectangular Subdivision” (NRS) has been introduced in .
Definition 3.2 (see ). Assumed that is the linear underestimated function of over the set , is the optimal solution of the underestimated problem of (), then a nested sequence is said to be normal if A rectangular subdivision process is said to be normal if any nested sequence of rectangles generated from the process is normal.
If with a given sufficient small number, then (when ) or (when ) is an -approximate global optimal solution of the problem (). Otherwise, the problem () will be divided into two subproblems: where is decided by the NRS process such as - subdivision.
Similar to the problem (), we can get the underestimated quadratic programming (noted as () and ()) for each of the subproblem () and () by replacing the concave part by their respective underestimated function and :(1)If either of them is infeasible, then the corresponding subproblem () or () is infeasible, and we will delete it.(2)If at least one subproblem is feasible, we can get the optimal solution (or ) of the underestimated subproblem () (or ()) for the subproblem () (or ()). Let upper bound , then delete the subproblem () or () of which the lower bound or is larger than .
Remarkably, if or , Algorithm 2.1 is used for solving the subproblem () or (). The corresponding optimal solution is noted as or . The upper bound for the optimal value will be updated, .
We delete those subproblems of which the lower bound are larger than . Then we select one subproblem from () and (), which has a smaller lower bound for optimal value, and divide it into two subproblems. Repeat this process until no subproblems exist.
In the following, we will give the detailed description of B&B-DCA algorithm.
Algorithm 3.3 (The Combined B&B-DCA Algorithm).
Set , give the tolerance a sufficient small number. Solve the underestimated problem () to obtain an optimal solution . Then use Algorithm 2.1 (DCA) to solve the problem (), the resulting optimal solution is noted as . Set problems set , upper bound , lower bound . . 2° Stop Criterion
Delete all with . Let be the set of remained subproblems. If , stop, and is an -global optimal solution of the problem (). Otherwise, goto 3°. 3° Branch
Select a problem () from the set of problems : with Then divide into and according to an NRS process, the relative subproblems are noted as () and (), set . 4° Bound
For the subproblem (), , solve the underestimated subproblem of () to obtain the optimal solutions . Let , . Then set and . 5° Deciding Whether to Call DCA Procedure
For . If , Algorithm 2.1 (DCA) is applied to solve the subproblem of (), the resulting solution is denoted by . Then set , and , goto 6°; otherwise, goto 6°. 6° Iteration
Let , and goto 2°.
Since the DCA method is an efficient local optimization method for DC programming, the combination of DCA and B&B algorithm will guarantee the global optimality and accelerate the convergence of general B&B algorithm (see ) for the problem (). Due to the decrease of upper bound , the convergence speed of the B&B algorithm will have some improvement. However, we do not need to implement the DCA in each subproblem. Only when some conditions are satisfied, the DCA procedure will be called so as to prevent from the overusing of DCA.
3.2. The Convergence of B&B-DCA Algorithm
Proof. If the algorithm terminates at finite iterations , is a global optimal solution of the problem () from the definition of and .
If the algorithm does not stop at finite iterations, it must generate an infinite nested sequence of rectangles. From the definitions of the upper bound and the lower bound , we know that the sequence is nonincreasing and a nonnegative number. Since the rectangle bisection process satisfies an NRS process, we have the following expression: Obviously, this means that Then we have Furthermore, , so the sequence generated by the algorithm above converges to a global optimal solution as .
We can see that NRS process plays an important role in the convergence of the B&B-DCA algorithm
4. Computational Tests
In this section, we will test the performance of proposed B&B-DCA algorithm for randomly generated datasets and the results will be compared with that of general B&B algorithm (see ) for problem (). Specifically, portfolio selection problem with concave transaction costs is also studied in Section 4.2. All the computational tests are coded by MATLAB (CPLEX is integrated to solve the relative quadratic programming) and run on a personal computer with Pentium Pro Dual 2.66 GHZ and 2 GB memory.
4.1. Problems with Randomly Generated Datasets
Datasets with different dimensions will be generated to test the performance of the B&B-DCA and general B&B algorithms. We will conduct numerical experiments of the proposed algorithms with dimensions from 50 to 400 for the problem (). In the following, we will give the generating process of the data sets and values of some parameters.
For the objective function in the problem (), the separable logarithm function is used to specify the concave part because of its wide applications in economic and financial problems. Let where and are randomly generated in regions [2, 3] and [3, 5], respectively, in uniform distribution.
For the convex part , we generate sequence in region [−1,1] in uniform distribution, and the covariance matrix of which is a positive definite matrix is noted as . So, we can ensure that the function is convex. The coefficients are randomly generated in regions [−1,1], respectively, in uniform distribution.
B&B-DCA and general B&B algorithms  are used to solve the problem () on the same datasets and parameters from dimension 50 to 400. In order to make the results more reliable, we will generate randomly five datasets to test the proposed B&B-DCA and general B&B algorithms for each dimension. The stop criterion of the algorithms is either obtaining an -optimal solution or the branch number greater than 20000.
The NRS process has an important effect on the convergence of the B&B-DCA and general B&B algorithms. To our knowledge, the NRS process includes the exhaustive bisection, -bisection, adaptive bisection, and largest distance bisection . Some evidences in [2, 5] show that the -bisection and largest distance bisection are greater than the other two methods. Since the calculation of -bisection is much more simple and have similar performance to the largest distance bisection, -bisection is applied in our proposed B&B-DCA and general B&B algorithms for problem ().
Results and Analysis
We find that optimal value computed by two algorithms are equal. In the following, we show the average branch number (Avg Bran) and mean CPU times (Time) for each dimension by the B&B-DCA and general B&B algorithms in Table 1. The average number of calling DCA process (Num DCA) in the B&B-DCA algorithm is also given. Furthermore, we test the performance of DCA applied to the problem (). The number of times (Num glob) when a global optimal solution is obtained after one process of DCA is also given in Table 1. Average CPU times (Time) for one process of DCA method is also present.
From the results and Table 1, we have the following comments.(i)General B&B and the proposed B&B-DCA algorithms can efficiently solve the problem () from the dimension 50 to 400. Due to the nice features of DCA (inexpensiveness, reliability, robustness, globality of computed solutions, etc.), the proposed B&B-DCA algorithm shows great superiority than general B&B algorithm not only in the average branch number but also in mean CPU time. Take dimension , for example, the average branch number by general B&B is 988, however the one by B&B-DCA is only 666.6. Then, the relative mean CPU time has a decrease of nearly 600 seconds by the B&B-DCA algorithm. So, embedding DCA into the B&B algorithm is quite necessary.(ii)The DCA method always gives a good approximation for optimal solution of problem () within short CPU time. It can be seen that it gives a global optimal solution five times during five computational tests for the dimension from 50 to 300. Even so, embedding DCA into B&B algorithm is necessary to guarantee the global optimal solution. Furthermore, if we are able to give a new method to obtain a more tighter lower bound for the optimal valve in the bound process, the proposed B&B-DCA algorithm can compute global optimal solution within a shorter time.
4.2. Portfolio Selection with Concave Transaction Costs
In this subsection, the proposed B&B-DCA and general B&B algorithms are applied to solve portfolio selection problem with concave transaction costs. It is pointed that concave transaction costs function is more reliable [4, 5]. The Mean-Variance (M-V) model can be written as : Vector is the decision portfolio with the investment weight in each asset. denotes the expected return rate and is the covariance matrix of the return rate for assets. Then, gives the risk (Variance) and gives net return of the portfolio , where denotes the nondecreasing concave transaction costs function, and is curved in Figure 1.
The sum of investment weight in each asset should be one, that is, , where denotes the vector with all entries equal to 1. , , and are the limitations of lower bound and upper bound of the investment . Parameter is the risk aversion index decided by the investor.
Tests will be performed on five datasets from the OR-Library (see ) which is a publicly available collection of test datasets for a variety of operations research problems. Each data set contains 290 weekly return rates for each stock. The data is computed from component stocks of Hang Seng, DAX, FTSE, S&P, and Nikkei Index for each dataset respectively. We can compute the relative expected return rate vector and covariance matrix . For each dataset, we will give the results for different value of from 0.05 to 0.95 (19 different values for ) in Table 2.
Similar to randomly generated datasets, we show average branch number (Avg Bran), average CPU time (Time), average number of calling DCA (Num DCA) for B&B and B&B-DCA, also total number (Num glob) when a global optimal solution is obtained after one process of DCA in Table 2.
As can be seen from Table 2, similar conclusions can be obtained to Table 1. First, the proposed B&B-DCA can accelerate the convergence of B&B to some extent in the aspects of branch number and CPU time. Second, DCA can compute a global optimal solution in short time and most cases. However, B&B is needed to confirm the globality of computed solutions. When a global solution is found, the loose lower bound for optimal value cannot guarantee fast convergence of the B&B algorithm. How to obtain a well-defined lower bound is a challenging and practical study.
Additionally, Figure 2 presents the efficient frontiers generated from the the M-V portfolio models without transaction costs (transaction costs function ) and with concave transaction costs ( is a separable nondecreasing concave function). No consideration of concave transaction costs will lead to inefficiently solutions. This will provide wrong guidance for the investors.
5. Conclusions and Future Research
In this paper, a class of DC programming is studied. General B&B is usually adopted to solve such problems. Based on existing local optimization method in DC programming, we have proposed a new global method B&B-DCA to solve the problem. DCA is an efficient local optimization method based on local optimality and the duality for solving DC programming, especially for large-scale problems.
Numerical tests on randomly generated datasets show that the proposed B&B-DCA has great superiority of branch number and computational time than general B&B algorithm with different dimensions. In addition, portfolio selection problem with transaction costs can be solved efficiently. The proposed B&B-DCA can be applied to solved other practical problems which can be modeled by this class of DC programming.
We find that DCA method always provides a global optimal solution, but the lower bound for the optimal value cannot guarantee fast convergence rate of B&B. If we can give a new method to obtain a more tighter lower bound, the proposed B&B-DCA algorithm can solve the problem with much shorter time. This seems significant in solving practical problems. Furthermore, other global optimization methods like filled function methods and so on can be combined with DCA to solve DC Programming. Some of these are under our current consideration.
This work is supported by National Natural Science Foundations of China: 10971162, 11101325, and 71171158.
J. E. Falk and R. M. Soland, “An algorithm for separable nonconvex programming problems,” Management Science, vol. 15, pp. 550–569, 1969.View at: Google Scholar
T. Q. Phong, L. T. H. An, and P. D. Tao, “On globally solving linearly constrained indefinite quadratic minimization problems by decomposition branch and bound method,” Operations Research, vol. 30, no. 1, pp. 31–49, 1996.View at: Google Scholar
P. D. Tao and L. T. H. An, “Convex analysis approach to d.c. programming: theory, algorithms and applications,” Acta Mathematica Vietnamica, vol. 22, no. 1, pp. 289–355, 1997.View at: Google Scholar
L. T. H. An, “Contribution à l'optimisation non convexe et à l'optimisation globale. Théorie, Algorithmes, Applications. Habilitation à Diriger des Recherches,” Université de Rouen, 1997.View at: Google Scholar
L. T. H. An, P. D. Tao, and M. Le Dung, “Exact penalty in d.c. programming,” Vietnam Journal of Mathematics, vol. 27, no. 2, pp. 169–178, 1999.View at: Google Scholar
R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, USA, 1970.
J. F. Toland, “On subdifferential calculus and duality in nonconvex optimization,” Société Mathématique de France. Bulletin. Mémoire, no. 60, pp. 177–183, 1979.View at: Google Scholar
R. Horst and H. Tuy, Global optimization, Springer-Verlag, Berlin, Second edition, 1993, Deterministic approaches.