A Hybrid Approach of Bundle and Benders Applied Large Mixed Linear Integer Problem

Abstract

Consider a large mixed integer linear problem where structure of the constraint matrix is sparse, with independent blocks, and coupling constraints and variables. There is one of the groups of constraints to make difficult the application of Benders scheme decomposition. In this work, we propose the following algorithm; a Lagrangian relaxation is made on the mentioned set of constraints; we presented a process heuristic for the calculation of the multiplier through the resolution of the dual problem, structured starting from the bundle methods. According to the methodology proposed, for each iteration of the algorithm, we propose Benders decomposition where quotas are provided for the value function and 𝜀-subgradient.

1. Introduction

The main objective of this work is to develop a methodology that combines the Benders decomposition and a heuristic for calculating the multiplier applied to a relaxed problem to solve a linear problem of large integer min𝑐^𝑡𝑥+𝑑^𝑡𝑦,s.t.𝐴𝑥+𝐵𝑦≤𝑏,𝑥∈𝑋_𝑅,𝑦∈𝑌,(1) where 𝑋_𝑅={𝑥∶𝐷𝑥≤𝑑,𝑥≥0} and 𝑌={𝑦∶𝐹𝑦≤𝑓,𝑦≥0,𝑦integer}.

Relaxing a part of the restrictions has been updated by multiplying the respective heuristic process, solving a local model of dual relaxed. As the algorithm presented, for each iteration with the multiplier obtained, apply iterations of Benders decomposition on the relaxed problem, obtaining an 𝜀-subgradient quotas and lower and upper optimal solution. The motivation of this work arises from an integer linear problem from the large expansion planning of the transmission of a digital telecommunication system in an urban area equivalent to a city the size of Rio de Janeiro/Brazil [1]. Section 2 gives an outline about decomposition methods for large scale, with all the relevant features for this work. Moreover, Section 3 shows the integer linear model in this work. More details of the methodology regularization and Benders decomposition are given in Section 4. Section 5 gives an improvement the Approximate Algorithm Bundle. In Section 6 is presented some conclusion and future works obtained from this work.

2. Decomposition Methods for Large Scale

The large mixed integer linear programming problem has highlighted the difficulty to be solved directly through commercial software. In such cases the Lagrangian, combined with subgradient optimization, is often used to lower levels to find the optimal value of the objective function. These quotes can be used, for example, the method of Branch-and-Bound [2], or just to measure the quality of feasible solutions. These properties are currently incorporated in commercial software [3]. Other strategies are also considered: obtaining upper bounds [4], more efficient routines on the generation of cuts and the use of parallel processing [5]. The Lagrangian was used by [6, 7] with its work on the traveling salesman problems, and methods of Branch-and-Bound and implicit enumeration had considerable gain in [8] with the Lagrangian, in [9]. There are several questions directed to the Lagrangian in integer linear problems, among them how to calculate the Lagrange multipliers, how to choose among the various relaxations of the problem, and how to obtain viable solutions to the primal problem. Techniques for solving the Lagrangian dual relaxation of combinatorial optimization problems in polynomial time by applying the algorithm as a subroutine of ellipsoids [10] have been presented with [11]. Other methodologies use heuristics decomposition lagrange combining the solution of the Lagrangian Dual by the method of subgradient, also considering the feasible solutions primal heuristics [12]. These techniques were applied to flow problems in networks “multicommodity” in [13], the capacitated location problems [14]. The decomposition Benders [15] is an exact method, finite, effective when the number of integer variables is much smaller than the number of continuous variables in which case the master problem has dimension much smaller than the original problem. However, for large problems, the Benders master problem can be difficult to solve because of the large size. It joins the convergence speed generally slow, making this method inefficient in many cases. Moreover, computational experiments have shown that a general code of Branch-and-Bound applied to solve the problem Benders master produces a tree often much larger than for solving the original problem. Thus, the disadvantage of this decomposition is often the difficulty in solving the master problem, making it inefficient.

Several papers were presented with the objective of solving the problem about master with a higher overall efficiency. Among these, [2, 9, 13, 21, 56, 62] implement the Benders decomposition with Lagrangian applied in cuts master problem. [22–25]. This transfers the difficulty of the master problem in solving iteratively the maximum dual function. In [26, 27] this method is denied due to lack of controllability (the optimal solution in the Benders master problem can never achieve the optimum in relaxed master problem) in the solution of the relaxed master problem. There are also suggestions on how to get a good initial set of cuts for the Benders master problem [28, 29]. In [30] suggests the use of linear relaxation for the Benders master problem in a number of initial iterations.

Motivated by these failures, [31, 32] developed the “Cross Decomposition” while exploiting the structures of primal and dual problems, combining the advantages of Dantzig-Wolfe decomposition [33, 34] and Benders [34, 35]. Reference [27] carried out a comparative study of several approaches to the problem Benders master, presenting an efficient method for solving a linear problem, the whole “Cross Decomposition” [31, 32, 36–39]. Theoretical aspects of Benders decomposition together with the “Cross Decomposition” are also discussed in [36, 40]. Changes in “Cross Decomposition” for integer linear programming problems were made by [13, 37]. These changes are made through the generalization of the method of Kornai and Liptak [41], which eliminates the need to use the master primal and dual problems. The dynamics of this decomposition is the subproblems, which iterates the primal and dual subproblems. Instead of using the last sub-problem solution as input to another, using an average of all previous subproblem solutions. The convergence proof of this methodology is found in [42]. For a certain class of location problems are presented structured exact solution methods from the “Cross Decomposition” in [36, 40, 43]. A comparison of techniques Kornai and Liptak for Decomposition and Cross-linear problems with block-angular structures and computational results is discussed in [16, 44]. Both methodologies have also been applied to problems of organizational planning [45]. Reference [46] presented a simplified algorithm of “Cross Decomposition” for multiple-choice right-side constraints. Applications involving stochastic transport problems were addressed in [47], involving comparative study with other methods.

The update of the multipliers can be made by various methods. If formulated as a linear problem, the simplex is traditionally used. Moreover, in general, a dual nondifferentiable and classical approach is the method of subgradient [1, 23–25, 48], which is known not to be a method of lowering. Although more complex, the techniques originally developed for bundle [49–51] are being increasingly used. The method exploits bundle data from previous iterations, vectors iterated, objective function, and subgradients, the bundle information, to produce new iteration. The method of 𝜀-descend [52] considers the method of programming differentiable subgradient conjugates [49, 50]. Kiwiel in [53–55] provides new insight into the method of Bundle based on classical methods of cutting planes developed by [56, 57]. The basic idea of generalization of cutting planes is to add a quadratic regularization to the linear approximation by convex parts to the objective function; this linearization is generated by using the subgradient. To avoid a large bundle, it is necessary to limit it. Reference [58], for example, presented a selection strategy based on the subgradient multipliers associated with the local model, where the bundle that remains in subgradient 𝑛+2, 𝑛 being the size of a variable of the problem, considered three approaches to specify the quadratic stabilization process, which are essentially equivalent. The first technique uses the confidence regions; see [59, 60]. The Moreau-Yosida regularization generates the proximal method used by [61]. A modern synthesis technique using bundle and metric variable is made ​​from the concept of Moreau-Yosida regularization in [62, 63]. Applications in control problems involving the method of bundle can be found in [64], and other applications using Lagrangian decomposition, networks, and comparative tests with other algorithms are developed in [60] and decompositions of large and parallel optimization in [65]. Lemarechal according to [66] “is not an exaggeration to say that 90 percent of the applications of nondifferentiability appear in decompositions of one form or another, while the remaining 10 percent are shown via the calculation of eigenvalues.” We mention also [67] when C. Lemaréchal says, “the nondifferentiable optimization has the biggest deficiency of the speed of convergence.”

3. Model

Consider the integer linear problem (P), motivated by an application in a telecommunications system [1], as follows: 𝜈_𝑃=min⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗,s.t.𝐷_𝑘𝑥_𝑘=𝑑_𝑘𝐹,𝑘=1,…,4,_𝑗𝑦_𝑗≤𝑓_𝑗𝐴,𝑗=1,2_𝑘𝑥_𝑘+𝐵_𝑘𝑦₁=𝐾_𝑘𝐶,𝑘=1,…,4,₃𝑥₃+𝐶₄𝑥₄+𝐶₂𝑦₂=𝐾₅,𝑥_𝑘≥0,𝑦_𝑗(P)≥0integer,𝑘=1,…,4𝑗=1,2, where the matrices 𝐴_𝑘,𝐵_𝑘,𝐶₂,𝐶₃,𝐶₄,𝐷_𝑘 and 𝐹_𝑗 have appropriate dimensions with the vectors 𝑐_𝑘,𝑑_𝑘,𝑒_𝑗,𝑓_𝑗,𝐾_𝑘,𝐾₅,𝑥_𝑘 and 𝑦_𝑗.

On the other hand, consider 𝑋=∏⁴_𝑘=1𝑋_𝑘 where 𝑋_𝑘={𝑥_𝑘;𝐷_𝑘𝑥_𝑘−𝑑_𝑘=0∧𝑥_𝑘≥0,𝑥_𝑘integers} and 𝑌=∏²_𝑗=1𝑌_𝑗 where 𝑌_𝑗={𝑦_𝑗;𝐹_𝑗𝑦_𝑗−𝑓_𝑗≤0∧𝑦_𝑗≥0,𝑦_𝑗integers}, supposed nonempty and limited, that is finite.

Consider the integer linear programming problem (P): 𝜈_𝑃𝐼=min⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗,s.t.𝐴_𝑘𝑥_𝑘+𝐵_𝑘𝑦₁=𝐾_𝑘𝐶,𝑘=1,…,4,₃𝑥₃+𝐶₄𝑥₄+𝐶₂𝑦₂=𝐾₅,()𝑥∈𝑋,𝑦∈𝑌.ILP A relaxation of the continuous variable 𝑥(ILP) generates ILP_x: 𝜈_𝑃=min⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗,s.t.𝐴_𝑘𝑥_𝑘+𝐵_𝑘𝑦₁=𝐾_𝑘𝐶,𝑘=1,…,4,₃𝑥₃+𝐶₄𝑥₄+𝐶₂𝑦₂=𝐾₅,𝑥∈𝑋_𝑅,𝑦∈𝑌,ILP_x where 𝑋_𝑅=∏⁴_𝑘=1𝑋_(𝑘)𝑅, and 𝑋_(𝑘)𝑅={𝑥_𝑘;𝐷_𝑘𝑥_𝑘−𝑑_𝑘=0∧𝑥_𝑘≥0}.

Also to relax the variable 𝑦  ILP_y is obtained is follows: 𝜈_𝑃𝑅=min⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗,s.t.𝐴_𝑘𝑥_𝑘+𝐵_𝑘𝑦₁=𝐾_𝑘𝐶,𝑘=1,…,4,₃𝑥₃+𝐶₄𝑥₄+𝐶₂𝑦₂=𝐾₅,𝑥∈𝑋_𝑅,𝑦∈𝑌_𝑅,ILP_y where 𝑌_(𝑗)={𝑦_𝑗;𝐹_𝑗𝑦_𝑗−𝑓_𝑗≤0∧𝑦_𝑗≥0}.

Relaxing the last block of constraints (ILP), we have the dual 𝑣_DI=max_𝜆𝜑(𝜆),D_I where for all 𝜆 defines the dual function. 𝜑(𝜆)=min_{(𝑥,𝑦)∈𝑊𝐼}ℓ𝑥,𝑦,𝜆=min⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑡𝐶₃𝑥₃+𝐶₄𝑥₄+𝐶₂𝑦₂−𝐾₅,s.t.𝑥,𝑦∈𝑊_𝐼,𝜑 where 𝑊_𝐼={(𝑥,𝑦);𝑥∈𝑋_𝑅,𝑦∈𝑌,𝐴_𝑘𝑥_𝑘+𝐵_𝑘𝑦₁=𝐾_𝑘,𝑘=1,…,4}.

The purpose of this relaxation is to ensure separability of blocks of variables 𝑥₃ and 𝑥₄, 𝑦₂ over in order, then applying the Benders decomposition.

4. Methodology Regularization and Benders Decomposition

The slow convergence of the algorithms in structured Benders decomposition applied large-scale integer linear programming problems motivated the development of the methodology to accelerate the classical method. Applies to Benders decomposition to the large-scale integer linear programming problem with a Lagrangian relaxation, to update the Lagrange multipliers by a Bundle Methods.

4.1. Benders Decomposition for the Relaxed Problem

The Benders decomposition applied to the relaxed problem 𝜑 is to reformulate this problem in an equivalent containing only 𝑦-integer variables and a continuous variable. Without loss of generality, assume that the problem has a finite optimal solution for all 𝜆.

For each 𝜆, 𝜑 can be rewritten as: min_𝑦∈𝑄²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑡𝐶₂𝑦₂+min⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+𝜆^𝑡𝐶₃𝑥₃+𝐶₄𝑥₄,𝐴_𝑘𝑥_𝑘=𝐾_𝑘−𝐵_𝑘𝑦₁,𝑘=1,…,4,𝑥∈𝑋_𝑅,𝜑^′ where 𝑄={𝑦∈𝑌;∃𝑥∈𝑋_𝑅suchthat𝐴_𝑘𝑥_𝑘=𝐾_𝑘−𝐵_𝑘𝑦₁𝑘=1,…,4}, nonempty.

For 𝑦∈𝑄 with 𝑦₁ fixed, the inner minimization subproblem (with explicit 𝑋_𝑅) 𝑣_𝐿=min_𝑥⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+𝜆^𝑡𝐶₃𝑥₃+𝐶₄𝑥₄,s.t.𝐷_𝑘𝑥_𝑘=𝑑_𝑘𝐴,𝑘=1,…,4,_𝑘𝑥_𝑘=𝐾_𝑘−𝐵_𝑘𝑦₁𝑥,𝑘=1,…,4,_𝑘(L)≥0,𝑘=1,…,4 has given its dual 𝑣_𝐷=max₍₎⁴_{𝑣,𝑢𝑖=1}𝑑^𝑡_𝑖𝑣_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢_𝑖,s.t.𝐷^𝑡_𝑖𝑣_𝑖+𝐴^𝑡_𝑖𝑢_𝑖≤𝑐_𝑖̃where𝑐_𝑖̃=𝑐_𝑖𝑐,𝑖=1,2,_𝑖+𝐶^𝑡_𝑖(D)𝜆,𝑖=3,4, where 𝑢=(𝑢₁,…,𝑢₄)^𝑡 and 𝑣=(𝑣₁,…,𝑣₄)^𝑡.

We assume that the polyhedral 𝑈(𝜆)=(𝑣,𝑢);𝐷^𝑡_𝑖𝑣_𝑖+𝐴^𝑡_𝑖𝑢_𝑖≤𝑐_𝑖̃where𝑐_𝑖̃=𝑐_𝑖𝑐,𝑖=1,2,_𝑖+𝐶^𝑡_𝑖𝜆,𝑖=3,4(2) are uniformly bounded, if necessary adding dimensions to variables (𝑣,𝑢) and 𝜆.

Thus we can define the set {(𝑣^𝑞,𝑢^𝑞)𝜆} for all 𝑞∈𝑃_𝑈(𝜆) (finite) of extreme points of 𝑈(𝜆). In this case, 𝜑^′ is equal to min_𝑦∈𝑄²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑡𝐶₂𝑦₂+max_{𝑞∈𝑃𝑈(𝜆)}⁴_𝑖=1𝑑^𝑡_𝑖𝑣^𝑞_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑞_𝑖.𝜑₁ Consider that 𝑧_𝐿(𝜆), the argument of the minimum, has for any subset 𝑃^′_𝑈(𝜆)⊆𝑃_𝑈(𝜆), the relaxed Benders master problem: 𝑧_𝐿(𝜆)=min_(𝑧,𝑦)𝑧,s.t.𝑧≥²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+⁴_𝑖=1𝑑^𝑡_𝑖𝑣^𝑞_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑞_𝑖,𝑧∈ℝ,𝑦∈𝑌,∀𝑞∈𝑃^′_𝑈(𝜆).(MB)

4.2. Quotas

For 𝜆 fixed, consider 𝑧_𝑈(𝜆)=∑²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑡𝐶₂𝑦₂+𝑣_𝐷 upper limit, where 𝑣_𝐷 was obtained in relaxed primal dual subproblem (D) and 𝑧_𝐿(𝜆) was obtained in relaxed primal dual subproblem (MB).

Then 𝑧_𝐿(𝜆)≤𝜑(𝜆)=min_{(𝑥,𝑦)∈𝑊𝐼}ℓ𝑥,𝑦,𝜆≤𝑧_𝑈(𝜆).(3) For 𝜆 variable, if we assume that, as done in the algorithm, the restrictions relaxed Benders master problem will remain the same from one to another iteration in 𝜆, then 𝑧_𝐿(𝜆^𝑝+1)≥𝑧_𝐿(𝜆^𝑝).

4.3. Regularization Dual Quadratic Problem

The iterative solution of the dual problem of maximizing into 𝜑D_I, that updates the multiplier 𝜆, is done using a local regulated model, such as the Bundle. However we do not know for each 𝜆, the value of 𝜑(𝜆), we only have lower quotas 𝑧_𝐿(𝜆) and upper 𝑧_𝑈(𝜆).

Suppose that we are in the 𝑝th iteration 𝜆^𝑝. Consider the model 𝑤𝜆^𝑝=max_𝑤,𝜌1𝑤−2𝑡_𝑝𝜌−𝜆^𝑝2,s.t.𝑤≤𝑔^𝑟𝑡𝜌−𝜆^𝑝+𝑧_𝑈𝜌^𝑟F,𝑟≥1,^I where 𝑡_𝑝>0, which determines the size of direction 𝜌−𝜆^𝑃.

For application of the Bundle method, consider the following. (a) The value of 𝑔^𝑟∶=𝐶₃𝑥^𝑟₃+𝐶₄𝑥^𝑟₄+𝐶₂𝑦^𝑟₂−𝐾₅ corresponds to some 𝜀_𝑟-subgradient of 𝜑 in 𝜆^𝑟.

Indeed, for (𝑥^𝑟,𝑦^𝑟)∈𝑊_𝐼 and any 𝜆^𝑝, 𝜑𝜆^𝑝=min_{(𝑥,𝑦)∈𝑊𝐼}ℓ𝑥,𝑦,𝜆^𝑝=ℓ𝑥^𝑟,𝑦^𝑟,𝜆^𝑝−𝜀_𝑟,forsome𝜀_𝑟≥0(4)

Defining ℓ, 𝜑𝜆^𝑝=𝜆^𝑝−𝜆^𝑡𝑔^𝑟+ℓ𝑥^𝑟,𝑦^𝑟,𝜆−𝜀_𝑟≥𝜆^𝑝−𝜆^𝑡𝑔^𝑟+𝜑(𝜆)−𝜀_𝑟,(5) that is, 𝑔^𝑟∈𝜕_𝜀𝑟𝜑(𝜆^𝑝) (subdifferencial of 𝜑(𝜆^𝑝)).(b) The linear cuts, corresponding to the local polyhedral model, and also 𝜑(𝜆^𝑟). This value is replaced by the upper bound 𝑧_𝑈(𝜌^𝑟), provided by the primal relaxed dual subproblem (D). To get 𝑔^𝑟, 𝑧_𝑈(𝜌^𝑟), 𝑧_𝐿(𝜌^𝑟) may require a few iterations of the Benders algorithm.

Indeed, consider acceptable (𝑔^𝑟,𝑧_𝑈(𝜌^𝑟),𝑧_𝐿(𝜌^𝑟)) if test quality of the approximation of 𝜑(𝜌^𝑟) is verified as follows: 𝑧_𝑈𝜌^𝑟−𝑧_𝐿𝜌^𝑟≤𝛼𝑧_𝑈𝜆^𝑝−1−𝑧_𝐿𝜆^𝑝−1,forsome0<𝛼<1.(6)

It is noted that the convergence Benders method ensures that the test will be checked in a finite number of iterations [15]. With this test it is ensured that the maximum error in calculating 𝜑, from one iteration to another in 𝜆, decreases. Indirectly we also expect 𝜀_𝑟→0.

With this set of information, we have the model “approximate” Ω_𝑝𝜌∶=min_𝑟𝑔^𝑟𝑡𝜌−𝜆^𝑝+𝑧_𝑈𝜌^𝑟.(7)

Thus, equivalently, F_I is as follows: max_𝜌Ω_𝑝𝜌−12𝑡_𝑝𝜌−𝜆^𝑝2.(8) The regularized model F_I has embedded in it the process (decomposed) plans secants and intends to determine a direction of ascent through the accumulated residue, with the approximate calculation of the dual function 𝜑(𝜆) in D_I, through Benders decomposition. The lemma and the proposition that follow seek to justify the existence and uniqueness of the solution of the quadratic subproblem, like it’s the aggregate subgradient.

Lemma 1 (see Lemma XV.3.1.1 in [69]). The problem (8) has a unique solution 𝜌^𝑝+1 characterized by 𝜌^𝑝+1=𝜆^𝑝+𝑡_𝑝𝑔_𝑝,𝑔_𝑝∈𝜕Ω_𝑝𝜌^𝑝+1.(9) Furthermore Ω(𝜆)≤𝑧_𝐿𝜆_𝑝+𝑔_𝑝^𝑡𝜆−𝜆_𝑝+𝑒_𝑝̂,∀𝜆,(10) where 𝑒_𝑝̂=Ω𝜌^𝑝+1−𝑧_𝐿𝜆_𝑝−𝑡_𝑝𝑔_𝑝².(11)

Proof. Suppose nonempty set generated by linear constraints, the existence and uniqueness of the solution 𝜌^𝑝+1 following the definition of a positive quadratic. The optimality condition for this solution is 0∈𝜕Ω_𝑝𝜌^𝑝+1−1𝑡_𝑝𝜌^𝑝+1+𝜆^𝑝.(12)
Then Ω(𝜆)≤Ω𝜌^𝑝+1+𝑔_𝑝^𝑡𝜆−𝜌^𝑝+1(13) which is equivalent to Ω(𝜆)≤𝑧_𝐿𝜆_𝑝+𝑔_𝑝^𝑡𝜆−𝜆_𝑝−𝑧_𝐿𝜆_𝑝+Ω𝜌^𝑝+1+𝑔_𝑝^𝑡𝜆_𝑝−𝜌^𝑝+1.(14) Considering (9) recognizes the expression (11) of 𝑒_𝑝̂.

Proposition 2 (see Lemma XV.3.1.2 in [69]). With the notation of Lemma 1, consider a quadratic function Ψ∶ℝ^𝑛→ℝ∪{∞} satisfying Ψ(𝜆)≤𝑧_𝐿𝜆_𝑝+𝑔_𝑝^𝑡𝜆−𝜆_𝑝+𝑒_𝑝̂=∶𝜑(𝜆),∀𝜆,(15) where equality in 𝜆=𝜌^𝑝+1. Then 𝜌^𝑝+1 maximizes the function Ψ(𝜆)(𝜆)+1∶=Ψ2𝑡_𝑝𝜆−𝜆^𝑝2.(16)

Proof. Applying (9) and (11) and defining relations Ψ can be written as follows: Ψ(𝜆)≤Ω𝜌^𝑝+1+𝑔_𝑝^𝑡𝜆−𝜌^𝑝+1−Ω𝜌^𝑝+1+𝑔_𝑝^𝑡𝜌^𝑝+1−𝜆_𝑝+𝑧_𝐿𝜆_𝑝+𝑒_𝑝̂,∴Ψ(𝜆)≤Ω𝜌^𝑝+1+𝑔_𝑝^𝑡𝜆−𝜌^𝑝+1(17) with equality in 𝜆=𝜌^𝑝+1. Subtracting the term (1/𝑡_𝑝)(𝜆−𝜆_𝑝)=0 both sides, Ψ(𝜆)≤Ω𝜌^𝑝+1+𝑔_𝑝^𝑡𝜆−𝜌^𝑝+1−12𝑡_𝑝𝜆−𝜆^𝑝2,(18) even with equality 𝜆=𝜌^𝑝+1. Now note that the function of the right side is maximized when 𝑔_𝑝−1𝑡_𝑝𝜆−𝜆_𝑝=0,(19) corresponding to 𝜌^𝑝+1, given by (9).

The function 𝜑(𝜆) is known to aggregate linearization approximation of 𝜑, where the limit on the model Ω, as described in Lemma 1.

A more convenient way to solve F_I is to be made through the dual problem. Define the Lagrangian and their optimality conditions as follows:

For 𝑑∶=𝜌−𝜆^𝑝, ℓ^∗1𝑤,𝑑,𝜂=𝑤−2𝑡_𝑝‖𝑑‖²−_{𝑟∈{1,…,ℓ)}𝜂^𝑟𝑤−𝑔^𝑟𝑡𝑑−𝑧_𝑈𝜌^𝑟ℓ,𝜂≥0_∗𝑤=0∴_{𝑟∈{1,…,ℓ)}𝜂^𝑟ℓ=1,_∗𝑑1=0∴−2𝑡_𝑝𝑑+_{{)𝑟∈1,…,ℓ}𝜂^𝑟𝑔^𝑟=0.(20) Complementarity, 𝜂_𝑟𝑤−𝑔^𝑟𝑡𝑑−𝑧_𝑈𝜌^𝑟=0,𝑟≥1.(21) Substituting these equations into ℓ^∗, it has the following quadratic linear problem: min℘𝜂,s.t._{𝑟∈{1,…,ℓ)}𝜂^𝑟𝜂=1,_𝑟≥0,DF_I where ℘𝜂=𝑡_𝑝2_{𝑟∈{1,…,ℓ)}𝜂^𝑟𝑔^𝑟2+_{𝑟∈{1,…,ℓ)}𝜂^𝑟𝑧_𝑈𝜌^𝑟.(22)

Consider that the optimality conditions have also to update the multiplier 𝜌=𝜆^𝑝+𝑡_{𝑝𝑟∈{1,…,ℓ)}𝜂^𝑟𝑔^𝑟,(23) where 𝜂 is single solution of DF_I.

5. The Approximate Algorithm of Bundle

5.1. Algorithm Partial Benders

At each iteration, the multiplier 𝜆 is used in subproblem (D), what, resolved, provides an upper bound 𝑧_𝑈 and generates a new Benders cut to be included in the relaxed master problem (MB). Solving this provides a lower limit 𝑧_𝐿 and a variable y for subproblem (L), which in turn is resolved in 𝑥. With 𝜆 fixed, this process is repeated and accumulated up all the cuts in the master problem Benders (MB), until the test (6) is satisfied. At the end of this process the values ​​of 𝑥, 𝑦, 𝑧_𝑈, and 𝑧_𝐿 are taken to the regularized model, a new update to the multiplier.

Note. We chose to include the quadratic model only the cut that corresponds to the test (6). However, we can include all cuts, leaving for future work the selection policy and proper disposal.

5.2. Approximate Test Armijo

One approach test Armijo [8] here will determine that the direction of the approximation increases 𝜑. Thereby, 𝛿_𝑝∶=Ω𝜌^𝑝+1−𝑧_𝐿𝜌^𝑝−12𝑡_𝑝𝜌^𝑝+1−𝜆^𝑝2,(24) where 𝜌^𝑝+1=𝜆^𝑝+𝑡_𝑝𝑔 and Ω is given by (7).

Approaching the values ​​of 𝜑 by lower quotas (𝑧_𝐿(𝜆)) and upper (𝑧_𝑈(𝜌)) has 𝜑𝜌^𝑝+1−𝜑𝜆^𝑝≤𝑧_𝑈𝜌^𝑝+1−𝑧_𝐿𝜆^𝑝.(25)

For 0<𝑚₁<1 provided, an approximation of the test Armijo will be satisfied in 𝜌^𝑝+1 if 𝑧_𝑈𝜌^𝑝+1−𝑧_𝐿𝜆^𝑝≥𝑚₁𝛿_𝑝,(26) where the left side is positive because 𝑧_𝑈𝜌^𝑝+1−𝑧_𝐿𝜆^𝑝≥𝑧_𝑈𝜌^𝑝+1−𝑧_𝐿𝜌^𝑝+1≥0.(27)

If we compare this to the test that corresponded to the exact calculation of the function 𝜑, observed that the difference between the current and the candidate has been replaced by an increase as much as 𝛿_𝑝 is an increase of the exact value. This expects that the test stop approximate Bundle method will not be anticipated, since also ensures a good approximation to the function 𝜑.

5.3. Regularization Algorithm for Updating the Multipliers with Relaxation

Before we present the algorithm and in order to keep the notation, replace the model F_I that is equivalent to 𝑤𝜆^𝑝=max_𝑤,𝜌1𝑤−2𝑡_𝑝𝜌−𝜆^𝑝2,s.t.𝑤≤𝑔^𝑟𝑡𝜌−𝜌^𝑟+𝑒_𝑟+𝑧_𝑈𝜆^𝑝6,𝑟≥1,^′ where 𝑒_𝑟∶=𝑒𝜆^𝑝,𝜌^𝑟,𝑔^𝑟∶=𝑧_𝑈𝜌^𝑟−𝑧_𝑈𝜆^𝑝+𝑔^𝑟𝑡𝜌^𝑟−𝜆^𝑝.(28)

We used without distinctions 𝑔(𝜌^𝑟)  and  𝑔^𝑟.

Algorithm 3. Initialization: They are given tolerance stop 𝛿≥0 and 𝜃>0. Consider ℓ>0 the maximum size of the Bundle, 𝑡₁>0. Get an initial dual feasible solution 𝜆¹, 𝑦⁰∈𝑌 and 𝑥⁰ initial feasible solution (L); that is, for 𝑦=𝑦⁰, 𝑥⁰ is solution of 𝐴_𝑘𝑥_𝑘=𝐾_𝑘−𝐵_𝑘𝑦₁𝑥,𝑘=1,…,4,_𝑘∈𝑋_𝑘,𝑘=1,…,4.(29) Calculate 𝑔¹=𝑔(𝜆¹). Make 𝑧_𝑈(𝜆¹)∶=ℓ(𝑥⁰,𝑦⁰,𝜆¹). Estimate 𝑧_𝐿(𝜆¹), for example, through an iteration of the Benders method. Choice 𝑚₁∈(0,1) reducing the test Armijo, 𝛼∈(0,1) is the reduction in quality test approximation 𝜑. Initialize the set of ascent 𝑃=𝜙, the accountant of iterations 𝑝=1, and the size of the bundle ℓ=1. For 𝑒₁=0, corresponding to the initial bundle (𝑔¹,𝑒₁), and the initial model 𝜌⟶Ω₁𝜌∶=𝑧_𝑈𝜆¹+𝑔^1𝑡𝜌−𝜆¹.(30)

Step 1 (principal calculation and test stop). Whether 𝜌^𝑝+1 is the unique solution of the quadratic problem such that 𝜌^𝑝+1=𝜆^𝑝+𝑡_𝑝𝑔^𝑝,where𝑔^𝑝∈𝜕Ω_𝑝𝜌^𝑝+1.(31)
Make 𝑒_𝑝̂∶=Ω_𝑝𝜌^𝑝+1−𝑧_𝐿𝜆^𝑝−𝑡_𝑝𝑔^𝑝2,𝛿_𝑝∶=Ω_𝑝𝜌^𝑝+1−𝑧_𝐿𝜆^𝑝−𝑡_𝑝2𝑔^𝑝2.(32) Calculate through of the algorithm 𝑧_𝑈𝜌^𝑝+1,𝑧_𝐿𝜌^𝑝+1,𝑔𝜌^𝑝+1.(33) If 𝛿_𝑝≤𝛿 and 𝑧_𝑈(𝜌^𝑝+1)−𝑧_𝐿(𝜌^𝑝+1)>𝜃, stop.

Step 2 (approximation test Armijo). If 𝑧_𝑈(𝜌^𝑝+1)−𝑧_𝐿(𝜆^𝑝)≥𝑚₁𝛿_𝑝, 𝑚₁∈(0,1) is “serious step”; otherwise, it is “null step,” check to Step 4.

Step 3 (serious step).  Make 𝜆^𝑝+1=𝜌^𝑝+1.  Add 𝑝 the set 𝑃; for 𝑟=1,…,ℓ. Permute 𝑒_𝑟 and 𝑒_𝑝̂ by, respectively, 𝑒_𝑟+𝑧_𝑈𝜆^𝑝−𝑧_𝑈𝜆^𝑝+1+𝑔^𝑟𝑡𝜆^𝑝−𝜆^𝑝+1,𝑒_𝑟̂+𝑧_𝑈𝜆^𝑝−𝑧_𝑈𝜆^𝑝+1+𝑔^𝑟𝑡𝜆^𝑝−𝜆^𝑝+1.(34)

Step 4 (control of bundle size). If ℓ=ℓ, then eliminate at least two of the bundle elements and insert the element (𝑔^𝑝,𝑒_𝑝̂).
Consider (𝑔^𝑡,𝑒_𝑡)_{𝑡=1,…,ℓ} the new bundle obtained (withℓ=ℓ).

Step 5. Insert (𝑔^ℓ+1,𝑒_ℓ+1) to the bundle, where 𝑒_ℓ+1=0 in the case of serious step, and in the case of null step, 𝑒_ℓ+1=𝑧_𝑈𝜌^𝑝+1−𝑧_𝑈𝜆^𝑝+𝑔^ℓ+1𝑡𝜌^𝑝+1−𝜆^𝑝.(35) Replace ℓ by ℓ+1, and update the model 𝜌→Ω_𝑝+1(𝜌)∶=min_𝑟{(𝑔^𝑟)^𝑡(𝜌−𝜆^𝑝+1)+𝑧_𝑈(𝜌^𝑟)}.

Step 6. Make 𝑝=𝑝+1, and return to Step 1.

5.4. Partial Benders Algorithm for Integer Linear Problem with Relaxation

Initialization: make 𝑞=1.

Step 1. Solve 𝑣_𝐷=max_{𝑣𝑖,𝑢𝑖}⁴_𝑖=1𝑑^𝑡_𝑖𝑉_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢_𝑖,s.t.𝐷^𝑡_𝑖𝑣_𝑖+𝐴^𝑡_𝑖𝑢_𝑖≤𝑐_𝑖̃,where𝑐_𝑖̃=𝑐_𝑖𝑐,𝑖=1,2,_𝑖+𝐶^𝑡_𝑖𝜌^𝑝+1,𝑖=3,4.(36) If there is no solution, stop: 𝜑₁ has no feasible solution. Otherwise, (𝑣^(𝑝,𝑞),𝑢^(𝑝,𝑞)) solution, and then 𝑧_𝑈𝜌^𝑝+1=²_𝑗=1𝑒^𝑡_𝑖𝑦_𝑗+𝜌^𝑡𝐶₂𝑦₂+𝑣_𝐷.(37) Generate a new constraint (cut) from (𝜌^𝑝+1,𝑣^(𝑝,𝑞), and 𝑢^(𝑝,𝑞)). Continue with Step 2.

Step 2. Solve min𝑧,s.t.𝑧≥²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜌^𝑝+1𝑡𝐶₂𝑦₂+⁴_𝑖=1𝑑^𝑡_𝑖𝑣^𝑞_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦^𝑡,𝑧∈ℝ,𝑦∈𝑌,∀𝑞.(38)
Consider (𝑧_𝐿(𝜌^𝑝+1),𝑦^𝑝+1) the optimal solution. Continue with Step 3.

Step 3. Solve min⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+𝜌^𝑝+1𝑡𝐶₃𝑥₃+𝐶₄𝑥₄.s.t.𝐴_𝑘𝑥_𝑘=𝑘_𝑘−𝐵_𝑘𝑦₁^𝑝+1𝑥,𝑘=1,…,4._𝑘∈𝑋_𝑅,𝑘=1,…,4.(39)
Whether 𝑥^𝑝+1 solution; continue with Step 4.

Step 4 (test quality approach 𝜑).  If 𝑧_𝑈𝜌^𝑝+1−𝑧_𝐿𝜌^𝑝+1≤𝛼𝑧_𝑈𝜆^𝑝−𝑧_𝐿𝜆^𝑝(40) end.
Otherwise, make 𝑝+1=𝑝, 𝑞=𝑞+1, and return to Step 1.

Remarks(1)The test for stopping the algorithm adds to the usual tolerance 𝛿 of bundles, the requirement that the function approximation is reasonable. In fact, for crude approximations of 𝜑, it is possible to have false serious steps with error 𝛿 false small, hence the need for 𝜃-approximation.(2)The Benders relaxed master problem (MB) should have some heuristics for selecting cuts, given that the accumulations of all inequalities explode the subproblem.

We present Figure 1 of the algorithm for the problem with integer linear relaxation.

Figure 1 

5.5. About Convergence

We opted to observe that 𝜃 small enough for the results cited correspond to guarantee the stability of the algorithm of bundle. This can be observed by adding a positive parameter 𝜃→0, the expression of errors linearization, and the gains predicted by the model (see, Lemma  3.2.1 in [69]. Thus if only guarantee the local convergence. Furthermore, the quality test approximation 𝜑₁ should be sufficient for obtaining convergence of the overall strength because the iterative process to arrive at the usual formulation of the bundle, with θ = 0. Undoubtedly, with the risk of being a high cost computational algorithm, as already noted. It presents the known result that guarantees no cycling algorithm Benders.

Theorem 4. The vectors composed of the vertices and their multipliers (𝑣^𝑝,𝑢^𝑝, and 𝜆^𝑝) generated at each iteration by the algorithm are different.

Proof. Suppose that the first (𝑝≥1) extreme points, say (𝑣¹,𝑢¹),(𝑣²,𝑢²),…,(𝑣^𝑝,𝑢^𝑝) the problem generated (D)  (𝜆) and 𝜆¹,…,𝜆^𝑝 obtained problem regularized F_I.
Then with the Step 2, it has min𝑧,s.t.𝑧≥²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑠𝑡𝐶₂𝑦₂+⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑝_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑝_𝑖,𝑧∈ℝ,𝑦∈𝑌,𝑠≤𝑝.(41) The optimal solution of this problem 𝑧, 𝑦, that is, for some 𝑘  (1≤𝑘≤𝑝), 𝑠=1,…,𝑝𝑧=²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗𝜆^𝑘𝑡𝐶₂𝑦₂+⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑘_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑘_𝑖≥²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑠𝑡𝐶₂𝑦₂+⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑠_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑠_𝑖.(42) As 𝑧 is a lower bound on the optimal cost primal relaxed 𝜑, 𝜑≥𝑧, and with (42), ⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑘_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑘_𝑖≤𝜑−²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑘𝑡𝐶₂𝑦₂.(43) On the other hand, in the next iteration of (D)(𝜆^𝑝+1) the solution (𝑣^𝑝+1,𝑢^𝑝+1) is a vertex 𝑈(𝜆). Then ⁴_𝑖=1𝑏^𝑡_𝑖𝑣_𝑖^𝑝+1+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢_𝑖^𝑝+1=⁴_𝐾=1𝑐^𝑡_𝑘𝑥_𝑘+𝜆^𝑝+1𝑡𝐶₃𝑥₃+𝐶₄𝑥₄,(44) where 𝑥 is a solution of (L)(𝜆).
As (𝑥,𝑦) is a viable solution 𝜑 it has ²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+𝜆^𝑝+1𝑡𝐶₂𝑦₂+⁴_𝐾=1𝑐^𝑡_𝑘𝑥_𝑘+𝜆^𝑝+1𝑡𝐶₃𝑥₃+𝐶₄𝑥₄≥𝜑.(45) Equivalently 𝜑−²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗≤⁴_𝐾=1𝑐^𝑡_𝑘𝑥_𝑘+𝜆^𝑝+1𝑡𝐶₃𝑥₃+𝐶₄𝑥₄+𝜆^𝑝+1𝑡𝐶₂𝑦₂.(46) Combining (43) and (44), it has ⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑘_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑘_𝑖+𝜆^𝑘𝑡𝐶₂𝑦₂≤𝜑−²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗≤⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+𝜆^𝑝+1𝑡𝐶₃𝑥₃+𝐶₄𝑥₄+𝜆^𝑝+1𝑡𝐶₂𝑦₂=⁴_𝑖=1𝑏^𝑡_𝑖𝑣_𝑖^𝑝+1+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢_𝑖^𝑝+1+𝜆^𝑝+1𝑡𝐶₂𝑦₂.(47) If 𝜑−(∑⁴_𝑘=1𝑐^𝑡_𝑘𝑥_𝑘+(𝜆^𝑝+1)^𝑡(𝐶₃𝑥₃+𝐶₄𝑥₄))=∑²_𝑗=1𝑒^𝑡_𝑗𝑦_𝑗+(𝜆^𝑝+1)^𝑡𝐶₂𝑦₂ then (𝑥,𝑦) solves the relaxed integer linear problem 𝜑.
Otherwise, ⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑘_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑘_𝑖+𝜆^𝑘𝑡𝐶₂𝑦₂<⁴_𝑖=1𝑏^𝑡_𝑖𝑣_𝑖^𝑝+1+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢_𝑖^𝑝+1+𝜆^𝑝+1𝑡𝐶₂𝑦₂(48) in which case (𝑣^𝑘,𝑢^𝑘,𝜆^𝑘)≠(𝑣^𝑝+1,𝑢^𝑝+1,𝜆^𝑝+1).
But inequality (42) is as follows: ⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑘_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑘_𝑖+𝜆^𝑘𝑡𝐶₂𝑦₂≥⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑠_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑠_𝑖+𝜆^𝑝+1𝑡𝐶₂𝑦₂,𝑠=1,…,𝑝.(49) On the other hand, of (48), ⁴_𝑖=1𝑏^𝑡_𝑖𝑣^𝑠_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^𝑠_𝑖+𝜆^𝑠𝑡𝐶₂𝑦₂<⁴_𝑖=1𝑏^𝑡_𝑖𝑣_𝑖^𝑝+1+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢_𝑖^𝑃+1+𝜆^𝑝+1𝑡𝐶₂𝑦₂,𝑠=1,…,𝑝,(50) and therefore (𝑣^𝑝+1,𝑢^𝑝+1,𝜆^𝑝+1)≠(𝑣^𝑠,𝑢^𝑠,𝜆^𝑠) 𝑠=1,…,𝑝.

Corollary 5. If 𝑞>1, the 𝑚th iteration internal, then 𝑦^𝑚+1≠𝑦^𝑚.

Proof. Suppose the contrary, that to solve (MB) with 𝑚 cuts, the solution 𝑦 is repeated. In this case, to solve the problem (D), we would obtain a vector (𝑣^𝑚+1,𝑢^𝑚+1) satisfying ⁴_𝑖=1𝑑^𝑡_𝑖𝑣^ℓ_𝑖+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢^ℓ_𝑖=⁴_𝑖=1𝑑^𝑡_𝑖𝑣_𝑖^𝑚+1+⁴_𝑖=1𝐾_𝑖−𝐵_𝑖𝑦₁^𝑡𝑢_𝑖^𝑚+1(51) for some ℓ={1,…,𝑚}. However, this only occurs when the optimality criterion is reached.

6. Conclusion and Future Works

Our main goal was to present an alternative technique using Lagrangian relaxation in solving a problem in integer linear programming. The work introduced a new algorithm structured from a block of relaxation of constraints that the problem presents difficulties when approached by traditional techniques Benders. We hope to take advantage of the computational process smoothing over other heuristic algorithms (Dantzig-Wolfe, subgradient) because its search direction is determined by processes similar to bundle method, which has shown proven results superior to those in many large problems [60]. It seems also unlikely that the technique of “Cross Decomposition” would be adaptable. As future works, we investigate other applications in order to verify the efficiency of the methods on structured problems and extend the decomposition to nonlinear problems and integer nonlinear, using the Lagrangian heuristic process along with the regularization. It is expected that other hybrid methodologies [71–74] can be applied in the solution of the problem (P) [1].

Acknowledgment

The authors are thankful to National Counsel of Technological and Scientific Development (CNPq).