A Memetic Lagrangian Heuristic for the 0-1 Multidimensional Knapsack Problem

Abstract

We present a new evolutionary algorithm to solve the 0-1 multidimensional knapsack problem. We tackle the problem using duality concept, differently from traditional approaches. Our method is based on Lagrangian relaxation. Lagrange multipliers transform the problem, keeping the optimality as well as decreasing the complexity. However, it is not easy to find Lagrange multipliers nearest to the capacity constraints of the problem. Through empirical investigation of Lagrangian space, we can see the potentiality of using a memetic algorithm. So we use a memetic algorithm to find the optimal Lagrange multipliers. We show the efficiency of the proposed method by the experiments on well-known benchmark data.

1. Introduction

The knapsack problems have a number of applications in various fields, for example, cryptography, economy, network, and so forth. The 0-1 multidimensional knapsack problem (0-1MKP) is an NP-hard problem, but not strongly NP-hard [1]. It can be considered as an extended version of the well-known 0-1 knapsack problem (0-1KP). In the 0-1KP, given a set of objects, each object that can go into the knapsack has a size and a profit. The knapsack has a certain capacity for size. The objective is to find an assignment that maximizes the total profit not exceeding the given capacity. In the case of the 0-1MKP, the number of capacity constraints is more than one. For example, the constraints can be a weight besides a size. Naturally, the 0-1MKP is a generalized version of the 0-1KP.

Let 𝑛 and 𝑚 be the numbers of objects and capacity constraints, respectively. Each object 𝑖 has a profit 𝑣_𝑖, and, for each constraint 𝑗, a capacity consumption value 𝑤_𝑗𝑖. Each constraint 𝑗 has a capacity 𝑏_𝑗. Then, we formally define the 0-1MKP as follows: maximize𝐯^𝑇𝐱subjectto𝑊𝐱≤𝐛,𝐱∈{0,1}^𝑛, (1) where 𝐯=(𝑣_𝑖) and 𝐱=(𝑥_𝑖) are 𝑛-dimensional column vectors, 𝑊=(𝑤_𝑗𝑖) is an 𝑚×𝑛 matrix, 𝐛=(𝑏_𝑗) is an 𝑚-dimensional column vector, and 𝑇 means the transpose of a matrix or a column vector. 𝑊, 𝐛, and 𝐯 are given, and each element of them is a nonnegative integer. In brief, the objective of the 0-1MKP is to find a binary vector 𝐱 which maximizes the weighted sum 𝐯^𝑇𝐱 satisfying 𝑚 linear constraints 𝑊𝐱≤𝐛.

For the knapsack problem with only one constraint, there have been a number of researches about efficient approximation algorithm to find a near-optimal solution. In this paper, we are interested in the problem with more than one constraint, that is, the multidimensional knapsack problem. In [2, 3] among others, the exact algorithms for 0-1MKP have been introduced. Heuristic approaches for 0-1MKP have also been extensively studied in the past [4–13]. Also, a number of evolutionary algorithms to solve the problem have been proposed [6, 14–19]. A number of methods for the 0-1 bi-knapsack problem, which is a particular case of 0-1MKP, have also been proposed. The reader is referred to [20–22] for deep surveys of 0-1MKP.

However, most researches directly deal with the discrete search space. In this paper, we transform the search space of the problem into a real space instead of directly managing the original discrete space. The 0-1MKP is the optimization problem with multiple constraints. We transform the problem using the multiple Lagrange multipliers. However, we have a lot of limitations since the domain is not continuous but discrete. Lagrangian heuristics have been mainly used to get good upper bounds of the integer problems by the duality. To get good upper bounds, a number of researchers have studied dual solvings using Lagrangian duality, surrogate duality, and composite duality, and there was a recent study using cutting plane method for 0-1MKP. However, in this paper, we focus on finding good lower bounds, that is, good feasible solutions.

There have been a number of studies about Lagrangian method for discrete problems. There were also a few methods that used Lagrange multipliers for 0-1MKP. Typically, most of them found just good upper bounds to be used in a branch-and-bound algorithm by dualizing constraints and hence could not find a feasible solution directly. To the best of the author’s knowledge, there have been only two Lagrangian methods to find lower bounds (feasible solutions). One is the constructive heuristic (CONS) proposed by Magazine and Oguz [10], and the other is its improvement using randomization (R-CONS) [13, 19]. There was also a method called LM-GA that improved the performance by combining CONS with genetic algorithms [23]. LM-GA used the real-valued weight-codings to make a variant of the original problem and then applied CONS (see Section 2.2 for details). LM-GA provided a new viewpoint to solve 0-1MKP, but it just used CONS for fitness evaluation and did not give any contribution in the aspect of Lagrangian theory.

In this paper, we present a local improvement heuristic for optimizing Lagrange multipliers. However, it is not easy to obtain good solutions by just using the heuristic itself. Through empirical investigation of Lagrangian space, we devise a novel memetic Lagrangian heuristic combined with genetic algorithms. The remainder of this paper is organized as follows. In Section 2, we present our memetic Lagrangian method together with some literature survey. In Section 3, we give our experimental results on well-known benchmark data, and we make conclusions in Section 4.

2. Lagrangian Optimization

2.1. Preliminaries

The 0-1MKP is a maximization problem with constraints. It is possible to transform the original optimization problem into the following problem using Lagrange multipliers: maximize𝐯^𝑇⟨{}𝐱−𝝀,𝑊𝐱−𝐛⟩subjectto𝐱∈0,1^𝑛. (2) It is easy to find the maximum of the transformed problem using the following formula: 𝐯^𝑇𝐱−⟨𝝀,𝑊𝐱−𝐛⟩=^𝑛_𝑖=1𝑥_𝑖𝑣_𝑖−^𝑚_𝑗=1𝜆_𝑗𝑤_𝑗𝑖+⟨𝝀,𝐛⟩. (3) To maximize the above formula for the fixed 𝝀, we have to set 𝑥_𝑖 to be 1 only if 𝑣_𝑖>∑^𝑚_𝑗=1𝜆_𝑗𝑤_𝑗𝑖 for each 𝑖. Since each 𝑣_𝑖 does not have an effect on the others, getting the maximum is fairly easy. Since this algorithm computes just ∑^𝑚_𝑗=1𝜆_𝑗𝑤_𝑗𝑖 for each 𝑖, its time complexity becomes 𝑂(𝑚𝑛).

If we only find out 𝝀 for the problem, we get the optimal solution of the 0-1MKP in polynomial time. We may have the problem that such 𝝀 never exists or it is difficult to find it although it exists. However, this method is not entirely useless. For arbitrary 𝝀, let the vector 𝐱 which achieves the maximum in the above formula be 𝐱^∗. Since 𝝀 is chosen arbitrarily, we do not guarantee that 𝐱^∗ satisfies the constraints of the original problem. Nevertheless, letting the capacity be 𝐛^∗=𝑊𝐱^∗ instead of 𝐛 makes 𝐱^∗ be the optimal solution by the following proposition [13, 19]. We call this procedure Lagrangian method for the 0-1MKP (LMMKP).

Proposition 1. The vector 𝐱^∗ obtained by applying LMMKP with given 𝝀 is the maximizer of the following problem: 𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒𝐯^𝑇𝐱𝑠𝑢𝑏𝑗𝑒𝑐𝑡𝑡𝑜𝑊𝐱≤𝐛^∗,𝐱∈{0,1}^𝑛. (4)

In particular, in the case that 𝜆_𝑘 is 0, the 𝑘th constraint is ignored. That is, 𝐱^∗ is the maximizer of the problems which have the capacities 𝐜’s such that 𝑐_𝑘≥𝑏^∗_𝑘 and 𝑐_𝑖=𝑏^∗_𝑖 for all 𝑖≠𝑘. In general, the following proposition [19] holds.

Proposition 2. In particular, if LMMKP is applied with 𝝀 such that 𝜆_𝑘1=0,𝜆_𝑘2=0,…, and   𝜆_𝑘𝑟=0, replacing the capacity 𝐛^∗ by 𝐜 such that 𝑐_𝑖=𝑏^∗_𝑖,𝑖𝑓𝑖≠𝑘_𝑗𝑐∀𝑗,_𝑖≥𝑏^∗_𝑖,𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (5) in Proposition 1 makes the proposition still hold.

Instead of finding the optimal solution of the original 0-1MKP directly, we consider the problem of finding 𝝀 corresponding to given constraints. That is, we transform the problem of dealing with 𝑛-dimensional binary vector 𝐱 into the one of dealing with 𝑚-dimensional real vector 𝝀. If there are Lagrange multipliers corresponding to given constraints and we find them, we easily get the optimal solution of the 0-1MKP. Otherwise we try to get the solution close to the optimum by devoting to find Lagrange multipliers which satisfy given constraints and are nearest to them.

2.2. Prior Work

In this subsection, we briefly examine existing Lagrangian heuristics for discrete optimization problems. Coping with nondifferentiability of the Lagrangian led to the last technical development: subgradient algorithm. Subgradient algorithm is a fundamentally simple procedure. Typically, the subgradient algorithm has been used as a technique for generating good upper bounds for branch-and-bound methods, where it is known as Lagrangian relaxation. The reader is referred to [24] for the deep survey of Lagrangian relaxation. At each iteration of the subgradient algorithm, one takes a step from the present Lagrange multiplier in the direction opposite to a subgradient, which is the direction of (𝐛^∗−𝐛), where 𝐛^∗ is the capacity obtained by LMMKP, and 𝐛 is the original capacity.

The only previous attempt to find lower bounds using Lagrangian method is CONS [10]; however, CONS without hybridization with other metaheuristics could not show satisfactory results. LM-GA by [23] obtained better results by the hybridization of weight-coded genetic algorithm and CONS. In LM-GA, a candidate solution is represented by a vector (𝑤₁,𝑤₂,…,𝑤_𝑛) of weights. Weight 𝑤_𝑖 is associated with object 𝑖. Each profit 𝑝_𝑖 is modified by applying several biasing techniques with these weights, that is, we can obtain a modified problem instance 𝑃^′ which has the same constraints as those of the original problem instance but has a different object function. And then, solutions for this modified problem instance are obtained by applying a decoding heuristic. In particular, LM-GA used CONS as a decoding heuristic. The feasible solutions for the modified problem instance are also feasible for the original problem instance since they satisfy the same constraints. So, weight-coding does not need an explicit repairing algorithm.

The proposed heuristic is different from LM-GA in that it improves CONS itself by using properties of Lagrange multipliers, but LM-GA just uses CONS as evaluation function. Lagrange multipliers in the proposed heuristic can move to more diverse directions than CONS because of its random factor.

2.3. Randomized Constructive Heuristic

Yoon et al. [13, 19] proposed a randomized constructive heuristic (R-CONS) as an improved variant of CONS. First, 𝝀 is set to be 𝟎. Consequently, 𝑥_𝑖 becomes 1 for each 𝑣_𝑖>0. It means that all positive-valued objects are put in the knapsack and so almost all constraints are violated. If 𝝀 is increased, some objects become taken out. We increase 𝝀 adequately for only one object to be taken out. We change only one Lagrange multiplier at a time. We randomly choose one number 𝑘 and change 𝜆_𝑘.

Reconsider (3). Making (𝑣_𝑖−∑^𝑚_𝑗=1𝜆_𝑗𝑤_𝑗𝑖) be negative by increasing 𝜆_𝑘 let 𝑥_𝑖=0 by LMMKP. For each object 𝑖 such that 𝑥_𝑖=1,  let 𝛼_𝑖 be the increment of 𝜆_𝑘 to make 𝑥_𝑖 be 0. Then, (𝑣_𝑖−∑^𝑚_𝑗=1𝜆_𝑗𝑤_𝑗𝑖−𝛼_𝑖𝑤_𝑘𝑖) has to be negative. That is, if we increase 𝜆_𝑘 by 𝛼_𝑖 such that 𝛼_𝑖>(𝑣_𝑖−∑^𝑚_𝑗=1𝜆_𝑗𝑤_𝑗𝑖)/𝑤_𝑘𝑖, the 𝑖th object is taken out. So, if we just change 𝜆_𝑘 to 𝜆_𝑘+min_𝑖𝛼_𝑖, leave 𝜆_𝑖 as it is for 𝑖≠𝑘, and apply LMMKP again, exactly one object is taken out. We take out objects one by one in this way and stop this procedure if every constraint is satisfied.

Algorithm 1 shows the pseudo code of R-CONS. The number of operations to take out the object is at most 𝑛, and computing 𝛼_𝑖 for each object 𝑖 takes 𝑂(𝑚) time. Hence, the total time complexity becomes 𝑂(𝑛²𝑚).

R-CONS(0-1MKP instance) {  𝝀←𝟎;  𝐼←{1,2,…,𝑛};  do   𝑘← random integer in [1,𝑚];   for  𝑖∈𝐼    𝛼𝑖←(𝑣𝑖−∑𝑚𝑗=1𝜆𝑗𝑤𝑗𝑖)/𝑤𝑘𝑖;   𝜆𝑘←𝜆𝑘+min𝑖∈𝐼𝛼𝑖;   𝐼←𝐼⧵{argmin𝑖∈𝐼𝛼𝑖};   (𝐱∗,𝐛∗)=LMMKP(𝝀);  until  𝐱∗ satisfies all the constraints;  return  𝝀; }

Algorithm 1 

Randomized constructive heuristic [13, 19].

2.4. Local Improvement Heuristic

Our goal is to improve a real vector 𝝀 obtained by R-CONS whose corresponding capacity is quite close to the capacity of the given problem instance. To devise a local improvement heuristic, we exploited the following proposition [13, 19].

Proposition 3. Suppose that 𝝀 and 𝝀^′ correspond to {𝐱,𝐛} and {𝐱^′,𝐛^′} by the LMMKP, respectively. Let 𝝀=(𝜆₁,𝜆₂,…,𝜆_𝑚) and 𝝀^′=(𝜆^′₁,𝜆^′₂,…,𝜆^′_𝑚), where 𝜆_𝑖=𝜆^′_𝑖  for 𝑖≠𝑘 and 𝜆_𝑘≠𝜆^′_𝑘. Then, if  𝜆_𝑘<𝜆^′_𝑘, 𝑏_𝑘≥𝑏^′_𝑘, and if 𝜆_𝑘>𝜆^′_𝑘,  𝑏_𝑘≤𝑏^′_𝑘.

Let 𝐛 be the capacity of the given problem instance and let 𝐛^∗ be the capacity obtained by LMMKP with 𝝀. By the above theorem, if   𝑏^∗_𝑘>𝑏_𝑘, choosing 𝝀^′=(𝜆₁,…,𝜆^′_𝑘,…,𝜆_𝑚) such that 𝜆^′_𝑘>𝜆_𝑘 and applying LMMKP with 𝝀^′ makes the value of  𝑏^∗_𝑘 smaller. It makes the 𝑘th constraint satisfied or the exceeded amount for the 𝑘th capacity decreased. Of course, another constraint may become violated by this operation. Also, which 𝜆_𝑘 to be changed is at issue in the case that several constraints are not satisfied. Hence, it is necessary to set efficient rules about which 𝜆_𝑘 to be changed and how much to change it. If good rules are made, we can find out better Lagrange multipliers than randomly generated ones quickly.

We selected the method that chooses a random number 𝑘(≤𝑚) and increases or decreases the value of 𝜆_𝑘 by the above theorem iteratively. In each iteration, a capacity vector 𝐛^∗ is obtained by applying LMMKP with 𝝀. If 𝐛^∗≤𝐛, all constraints are satisfied and hence the best solution is updated. Since the possibility to find a better capacity exists, the algorithm does not stop here. Instead, it chooses a random number 𝑘 and decreases the value of 𝜆_𝑘. If 𝐛^∗≰𝐛, we focus on satisfying constraints preferentially. For this, we choose a random number 𝑘 among the numbers such that their constraints are not satisfied and increase 𝜆_k hoping the 𝑘th value of corresponding capacity to be decreased and then the 𝑘th constraint to be satisfied. We set the amount of 𝜆_𝑘’s change to be the fixed value 𝛿.

Most Lagrangian heuristics for discrete problems have focused on obtaining good upper bounds, but this algorithm is distinguished in that it primarily pursues finding feasible solutions. Algorithm 2 shows the pseudo code of this local improvement heuristic. It takes 𝑂(𝑛𝑚𝑁) time, where 𝑁 is the number of iterations.

LocalImprovementHeuristic(𝝀, 0-1MKP instance) {  for  𝑖←1 to  𝑁   (𝐱∗,𝐛∗)←LMMKP(𝝀);   𝐼←{𝑖∶𝑏∗𝑖≤𝑏𝑖} and 𝐽←{𝑖∶𝑏∗𝑖>𝑏𝑖};   if  𝐼={1,2,…,𝑚}    update the best solution;    choose a random element 𝑘 among 𝐼;    𝜆𝑘←𝜆𝑘−𝛿;   else    choose a random element 𝑘 among 𝐽;    𝜆𝑘←𝜆𝑘+𝛿;  return  𝝀 corresponding to the best solution; }

Algorithm 2 

Local improvement heuristic.

The direction by our local improvement heuristic looks similar to that by the subgradient algorithm described in Section 2.2, but the main difference lies in that, in each iteration, the subgradient algorithm changes all coordinate values by the subgradient direction but our local improvement heuristic changes only one coordinate value. Consequently, this could make our local improvement heuristic find lower bounds more easily than subgradient algorithm usually producing upper bounds.

2.5. Investigation of the Lagrangian Space

The structure of the problem space is an important factor to indicate the problem difficulty, and the analysis of the structure helps efficient search in the problem space [25–27]. Recently Puchinger et al. [28] gave some insight into the solution structure of 0-1MKP. In this subsection, we conduct some experiments and get some insight into the global structure of the 0-1MKP space.

In Section 2.1, we showed that there is a correspondence between binary solution vector and Lagrange multiplier vector. Strictly speaking, there cannot be a one-to-one correspondence in the technical sense of bijection. The binary solution vector has only binary components, so there are only countably many such vectors. But the Lagrange multipliers are real numbers, so there are uncountably many Lagrange multiplier vectors. Several multiplier vectors may correspond to the same binary solution vector. Moreover, some multiplier vectors may have multiple binary solution vectors.

Instead of directly finding an optimal binary solution, we deal with Lagrange multipliers. In this subsection, we empirically investigate the relationship between binary solution space and Lagrangian space (i.e., {𝐱_𝝀s} and {𝝀s}).

We made experiments on nine instances (𝑚⋅𝑛) changing the number of constraints (𝑚) from 5 to 30 and the number of objects from 100 to 500. We chose a thousand of randomly generated Lagrange multipliers and plotted, for each pair of Lagrange multiplier vectors, the relation between the Hamming distance in binary solution space and the Euclidean distance in Lagrangian space. Figure 1 shows the plotting results. The smaller the number of constraints (𝑚) is, the larger the Pearson correlation coefficient (𝜌) is. We also made the same experiments on locally optimal Lagrange multipliers. Figure 2 shows the plotting results. Locally optimal Lagrange multipliers show much stronger correlation than randomly generated ones. They show strong positive correlation (much greater than 0.5). It means that binary solution space and locally optimal Lagrangian space are roughly isometric. The results show that both spaces have similar neighborhood structures. So this hints that it is easy to find high-quality Lagrange multipliers satisfying all the capacity constraints by using memetic algorithms on locally optimal Lagrangian space. That is, memetic algorithms can be a good choice for searching Lagrangian space directly.

(a) 5.100: 𝜌=0.44

(b) 5.250: 𝜌=0.50

(c) 5.500: 𝜌=0.45

(d) 10.100: 𝜌=0.35

(e) 10.250: 𝜌=0.37

(f) 10.500: 𝜌=0.37

(g) 30.100: 𝜌=0.19

(h) 30.250: 𝜌=0.19

(i) 30.500: 𝜌=0.19

Figure 1 

Relationship between distances on solution space and those on Lagrangian space (among randomly generated Lagrangian vectors). ∗𝑥-axis: distance between Lagrangian vectors, 𝑦-axis: distance between binary solution vectors, and 𝜌: Pearson correlation coefficient.

(a) 5.100: 𝜌=0.91

(b) 5.250: 𝜌=0.96

(c) 5.500: 𝜌=0.97

(d) 10.100: 𝜌=0.80

(e) 10.250: 𝜌=0.92

(f) 10.500: 𝜌=0.96

(g) 30.100: 𝜌=0.60

(h) 30.250: 𝜌=0.80

(i) 30.500: 𝜌=0.91

Figure 2 

Relationship between distances on solution space and those on Lagrangian space (among locally optimal Lagrangian vectors). ∗𝑥-axis: distance between Lagrangian vectors, 𝑦-axis: distance between binary solution vectors, and 𝜌: Pearson correlation coefficient.

2.6. Proposed Memetic Algorithm

A genetic algorithm (GA) is a problem-solving technique motivated by Darwin’s theory of natural selection in evolution. A GA starts with a set of initial solutions, which is called a population. Each solution in the population is called a chromosome, which is typically represented by a linear string. This population then evolves into different populations for a number of iterations (generations). At the end, the algorithm returns the best chromosome of the population as the solution to the problem. For each iteration, the evolution proceeds in the following. Two solutions of the population are chosen based on some probability distribution. These two solutions are then combined through a crossover operator to produce an offspring. With low probability, this offspring is then modified by a mutation operator to introduce unexplored search space into the population, enhancing the diversity of the population. In this way, offsprings are generated and they replace part of or the whole population. The evolution process is repeated until a certain condition is satisfied, for example, after a fixed number of iterations. A GA that generates a considerable number of offsprings per iteration is called a generational GA, as opposed to a steady-state GA which generates only one offspring per iteration. If we apply a local improvement heuristic typically after the mutation step, the GA is called a memetic algorithm (MA). Algorithm 3 shows a typical generational MA.

create an initial population of a fixed size 𝑃; do  for  𝑖←1  to  𝑃/2   choose parent1 𝑖 and parent2 𝑖 from population;   𝑜𝑓𝑓𝑠𝑝𝑟𝑖𝑛𝑔𝑖 ← crossover(parent1 𝑖, parent2 𝑖);   mutation(𝑜𝑓𝑓𝑠𝑝𝑟𝑖𝑛𝑔𝑖);   apply a local improvement heuristic to 𝑜𝑓𝑓𝑠𝑝𝑟𝑖𝑛𝑔𝑖;  replace(population, {𝑜𝑓𝑓𝑠𝑝𝑟𝑖𝑛𝑔𝑖}); until (stopping condition) return the best individual;

Algorithm 3 

Framework of our memetic algorithm.

We propose an MA for optimizing Lagrange multipliers. It conducts search using an evaluation function with penalties for violated capacity constraints. Our MA provides an alternative search method to find a good solution by optimizing 𝑚 Lagrange multipliers instead of directly dealing with binary vectors with length 𝑛 (𝑚≪𝑛).

The general framework of an MA is used in our study. In the following, we describe each part of the MA.

Encoding. Each solution in the population is represented by a chromosome. Each chromosome consists of 𝑚 genes corresponding to Lagrange multipliers. A real encoding is used for representing the chromosome 𝝀.

Initialization. The MA first creates initial chromosomes using R-CONS described in Section 2.3. We set the population size 𝑃 to be 100.

Mating and Crossover. To select two parents, we use a random mating scheme. A crossover operator creates a new offspring by combining parts of the parents. We use the uniform crossover.

Mutation. After the crossover, mutation operator is applied to the offspring. We use a gene-wise mutation. After generating a random number 𝑟 from 1 to 𝑚, the value of each gene is divided by 𝑟.

Local Improvement. We use a local improvement heuristic described in Section 2.4. The number of iterations (𝑁) is set to be 30,000. We set 𝛿 to 0.0002.

Replacement and Stopping Condition. After generating 𝑃/2 offspring, our MA chooses the best 𝑃 individual among the total 3𝑃/2 ones as the population of the next generation. Our MA stops when the number of generations reaches 100.

Evaluation Function. Our evaluation function is to find a Lagrange multiplier vector 𝝀 that has a high fitness satisfying the capacity constraints as much as possible. In our MA, the following is used as the objective function to maximize, which is the function obtained by subtracting the penalty from the objective function of the 0-1MKP: 𝐯^𝑇𝐱^∗−𝛾_{𝐛∗𝑖>𝐛𝑖}𝐛^∗_𝑖−𝐛_𝑖, (6) where 𝛾 is a constant which indicates the degree of penalty, and we used a fixed value 0.7.

3. Experiments

We made experiments on well-known benchmark data publicly available from the OR-Library [29], which are the same as those used in [6]. They are composed of 270 instances with 5, 10, and 30 constraints. They have different numbers of objects and different tightness ratios. The tightness ratio means 𝛼 such that 𝑏_𝑗=𝛼∑^𝑛_𝑖=1𝑤_𝑗𝑖 for each 𝑗=1,2,…,𝑚. The class of instances are briefly described below.𝑚.𝑛−𝛼: 𝑚 constraints, 𝑛 objects, and tightness ratio 𝛼. Each class has 10 instances.

The proposed algorithms were implemented with gcc compiler on a Pentium III PC (997 MHz) using Linux operating system. As the measure of performance, we used the percentage difference-ratio 100×|𝐿𝑃_𝑜𝑝𝑡𝑖𝑚𝑢𝑚−𝑜𝑢𝑡𝑝𝑢𝑡|/𝑜𝑢𝑡𝑝𝑢𝑡 which was used in [6], where 𝐿𝑃_𝑜𝑝𝑡𝑖𝑚𝑢𝑚 is the optimal solution of the linear programming relaxation over ℝ. It has a value in the range of [0,100]. The smaller the value is, the smaller the difference from the optimum is.

First, we compared constructive heuristics and local improvement heuristic. Table 1 shows the results of CONS, R-CONS, and R-CONS-L, where R-CONS-L starts with a solution produced by R-CONS and locally improves it by the local improvement heuristic described in Section 2.4. We can see that R-CONS-L largely improves the results of R-CONS.

Table 1 

Results of local search heuristics on benchmark data.

Next, to verify the effectiveness of the proposed MA, we compared the results with a multistart method using R-CONS-L. Multistart R-CONS-L returns the best result from 5,000 independent runs of R-CONS-L. Table 2 shows the results of multistart R-CONS-L and our MA. The proposed MA outperformed multistart R-CONS-L.

Table 2 

Results of memetic Lagrangian heuristic on benchmark data.

The last column in Table 2 corresponds to the numbers of improved and equalled results compared with the best results of [6]. Surprisingly, the proposed memetic algorithm could find better results than one of the state-of-the-art genetic algorithms [6] for some instances with 30 constraints (5 best over 90 instances).

4. Concluding Remarks

In this paper, we tried to find good feasible solutions by using a new memetic algorithm based on Lagrangian relaxation. To the best of the author’s knowledge, it is the first trial to use a memetic algorithm when optimizing Lagrange multipliers.

The Lagrangian method guarantees optimality with the capacity constraints which may be different from those of the original problem. But it is not easy to find Lagrange multipliers that accord with all the capacity constraints. Through empirical investigation of Lagrangian space, we knew that the space of locally optimal Lagrangian vectors and that of their corresponding binary solutions are roughly isometric. Based on this fact, we applied a memetic algorithm, which searches only locally optimal Lagrangian vectors, to the transformed problem encoded by real values, that is, Lagrange multipliers. Then we could find high-quality Lagrange multipliers by the proposed memetic algorithm. We obtained a significant performance improvement over R-CONS and multistart heuristic.

In recent years, there have been researches showing better performance than Chu and Beasley [6] on the benchmark data of 0-1MKP [4, 5, 8, 9, 11–13]. Although the presented memetic algorithm did not dominate such state-of-the-art methods, this study could show the potentiality of memetic search on Lagrangian space. Because we used simple genetic operators, we believe that there is room for further improvement on our memetic search. The improvement using enhanced genetic operators tailored to real encoding, for example, [30], will be a good future work.

Acknowledgments

The present research has been conducted by the research grant of Kwangwoon University in 2013. This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) and funded by the Ministry of Education, Science and Technology (2012-0001855).