Abstract
01 Knapsack problem (KP) is NPhard. Approximate solution is vital for solving KP exactly. In this paper, a fast polynomial time approximate solution (FPTAS) is proposed for KP. FPTAS is a local search algorithm. The best approximate solution to KP can be found in the neighborhood of the solution of upper bound for exact kitem knapsack problem (EkKP) where k is near to the critical item s. FPTAS, in practice, often achieves high accuracy with high speed in solving KP. The computational experiments show that the approximate algorithm for KP is valid.
1. Introduction
01 Knapsack problem (KP) is a classical combinatorial optimization problem. KP is NPhard. Knapsack problems were used to model capital budgeting problems in which investment projects are to be selected subject to expenditure limitations. Additionally, the knapsack problem has been used to model loading problems. Knapsack problems, moreover, arise as suboptimization problems in solving larger optimization problem [1]. KP can be formulated as follows:where is the profit of item , is the weight of item , is the capacity of the knapsack, is the number of items in KP, and variable indicates whether item is selected or not. Without loss of generality, it is assumed that all items are arranged in nonincreasing order of efficiency.
It is very important to find a fast polynomial approximation for KP in practice. Greedy algorithm [1], an approximate algorithm for KP, is on the basis of certain rules, such as the item being selected with priority over larger efficiency, larger profit, or smaller weight. Greedy algorithm is O(n) time complexity, but it is not an ɛapproximate algorithm [1]. Approximation algorithm is often kneighborhood local search method. Increasing the radius k of the neighborhood can improve the accuracy of the approximate algorithm [1, 2]. While PTAS for KP typically require only O(n) storage, all FPTAS are based on dynamic programming and their memory requirement increases rapidly with the accuracy ɛ, which makes them impractical even for relatively big values of ɛ [3]. Heuristics rules are adopted to decrease the calculation in searching accurate approximation, such as harmony search algorithm [4, 5], amoeboid organism algorithm [6], cuckoo search algorithm [5, 7], binary monarch butterfly optimization [8], cognitive discrete gravitational search algorithm [9], bat algorithm [10], and wind driven optimization [11]. Nowadays, it is tend to combine different heuristics together in solving combinatorial optimization problem, such as mixedvariable differentiate evolution [12], selfadaptive differential evolution algorithm [13], twostage cooperative evolutionary algorithm [14], cooperative water wave optimization algorithm with reinforcement learning [15], and cooperative multistage hyperheuristic algorithm [16]. However, these algorithms cannot guarantee the accuracy of the solution of KP. Meanwhile, these methods for the solution to KP are generally timeconsuming. More domain knowledge is necessary for better algorithm. Pisinger gave an exact algorithm for KP [17] which is based on an expanding core. The items not included in the core are certain to be selected or not in the optimal solution, while the items in the core are uncertain to be selected or not in the optimal solution. He found that algorithms solving some kinds of core problem may be stuck by difficult cores [18]. For example, KP is determined by
It is easy to find that the core of KP is [1, 10000], so this problem is difficult to tackle by the exact algorithm in [17]. In this paper, it is easy to get an approximate solution of KP which objective value is 6395122580. The solution can be proved to be the optimal solution.
In this paper, a fast polynomial approximate solution is proposed for 01 knapsack problems based on the solution of its upper bound. Firstly, an upper bound is presented based on the exact kitem knapsack problem EkKP in Section 2. Secondly, an initial solution of KP is constructed on the basis of the solution of upper bound of EkKP in Section 3.1. Thirdly, the approximate solution is proposed to find the best solution in the neighborhood of the initial solution in Section 3.2. In Section 3.3, the best approximation solution is achieved by comparison with number of items changing. In Section 3.4, the calculation for the approximate solution of KP is analyzed. In Section 4, computational experiments of KP are implemented. The results show that the approximate solution proposed in this paper can achieve high accuracy in general. It implicates that the exact solution to KP is similar to the solution to the upper bound. The algorithm proposed here is a fast polynomial approximate solution to KP.
2. Upper Bound for KP
If there are exact objects selected in the knapsack, then KP is an exact kitem knapsack problem EkKP formulated as follows [3]:
The upper bound of EkKP may be achieved by Lagrangian relaxation of capacity constraint:
Suppose that
Then
So can be solved by sorting in descending order in O (nlnn) time if λ and k are fixed. The upper bound of EkKP is formulated as follows:
It can be proved that is a unimodal function of λ if k is fixed. B_{k} can be quickly solved by linear search algorithm [19]. So B_{k} can be solved in O (nlnn) time.
The upper bound of KP is the maximum value of B_{k}:
It can be proved that B_{k} increases when and decreases when , where the critical item s satisfies.
So the upper bound of KP is
Example 1. Consider the instance of KP listed in Table 1. The capacity is 467.8435.
Here the critical item s is 20. There is no feasible solution with 20 items included in the knapsack. So the upper bound = 0. If k is 19, we have the optimal ratio = 1.0003251 and the upper bound = 486.9565 by (8). The solution of is as follows:So the upper bound of KP is 486.9565.
We test different upper bounds of KP from Pisinger’s paper [20], where the weights and profits of items are randomized. The instances listed in Table2 are tested to compare our proposed method with the existing upper bounds, such as the upper bound U_{MT2} proposed by Martello and Toth [21], the improved upper bound U_{MTM} [22] and the upper bound U_{kmax} with maximum cardinality [23]. The difficult instances can be constructed as follows [20]:(i)Uncorrelated instances with similar weights: Weights are distributed in [R, R + 100] and the profits in [1, 1000].(ii)Uncorrelated data instances: and are chosen randomly in [1, R].(iii)Weakly correlated instances: Weights are chosen randomly in [1, R] and the profits in [ − 0.1 R, + 0.1 R] such that .(iv)Strongly correlated instances: Weights are distributed in [1, R] and = + 0.1R.(v)Inverse strongly correlated instances: Profits are distributed in [1, R] and = p_{i}+0.1R.(iv)Almost strongly correlated instances: Weights are distributed in [1, R] and the profits in [ + 0.098R, + 0.102R].(vii)Subset sum instances: Weights are randomly distributed in [1, R] and = .(viii)Circle instances circle (d): The weights are uniformly distributed in [1, R] and for each weight the corresponding profit is chosen as where d is 2/3.(ix)Profit ceiling instances pceil (d): The weights of the n items are randomly distributed in [1, R], and the profits are set to . The parameter d was chosen as d = 3.(x)Multiple strongly correlated instances mstr (k_{1}, k_{2}, d): The weights of the n items are randomly distributed in [1, R]. If the weight is divisible by d, then we set the profit : = + k_{1}; otherwise set it to : = + k_{2}. We set d: = 6 here.For each instance type, a series of K = 100 instances is performed, and the capacity is determined by (13). All the instances above are generated with data range R = 10^{3}, 10^{4}, 10^{5}, 10^{6} or 10^{7}.There are 100 instances for each type of KP where the capability is described as follows:The mean relative error of upper bounds to the best upper bound of KP is listed in Table 3.
From Table 3, we find that the relative error of the upper bound B is the minimum in general. The upper bounds of 5000 instances are carried out with different methods. More details are listed in Table 4.
From Table 4, we find that U_{kmax} is the best upper bound in 2025 instances. The upper bound B plays an important role in obtaining the best upper bound even if .
The upper bound can gather most items selected in the optimal solution all together. For example, KP is determined byA maximum of 631 items can be selected in the knapsack. The upper bound and the solution are as follows:The initial solution is equal to . The best solution in the neighborhood can be described as follows:It is obviously that there is little difference between and . Relation between and is displayed in Figure 1. Relation between and is displayed in Figure 2. It is found that changes more dramatically than . It makes solution to KP easy.
From Figure 1, we can see that of all items selected are larger, while the others are smaller. It is easy to get a better solution on the basis of than that on the basis of efficiency .
3. Approximate Solutions
The approximate solution to EkKP is a solution to KP. We obtain the best approximate solution to KP by comparison to approximate solutions for EkKP where k is near to the critical item s. In order to achieve a better solution to EkKP, we firstly obtain an initial solution on the basis of the solution to the upper bound of EkKP. Then the initial solution is developed by local search in the neighborhood of the initial solution. At last, the approximate solution to KP is the best approximate solution to EkKP with various k. The upper bound of EkKP is used to decrease calculation. The solution to the upper bound of EkKP and the upper bound make key contribution in FPTAS to KP.
3.1. Initial Solution to EkKP
Let be an optimal solution to the upper bound . We may obtain an initial solution to EkKP on the basis of . The initial solution to EkKP is determined by
In Example 1, is an initial solution and its objective value is
3.2. Approximate Solution to EkKP
Approximate solution to EkKP is the best solution in the neighborhood of . Let be a neighborhood of that is defined by
It is obvious that the size of increases with . But it is unnecessary to take into account all elements of . Algorithm 1 is a fast algorithm for searching the best solution in .
In order to decrease the calculation for the approximate solution to KP, is redefined by (21) and Step 1 in Algorithm 1 is modified correspondingly.where
From (21), we know that the size of is limited. In order to search for the best solution, it is unnecessary to seek all solutions in . Let
The best approximate solution corresponds to , so the calculation is O (nlnn) time when equals to 1. A better solution is generated if the profit sum of items selected in the knapsack is less than that of items not selected in the knapsack, and the capacity constraint still holds at the same time. On the other hand, the calculation decreases via variable reduction in practice.
In Example 1, let the approximate solution equal to and objective value equal to firstly, and then the approximate solution is updated by the best solution in if possible.
Similarly, is replaced by the best solution in if possible.
There is no better solution of KP in and . We get an approximate solution with 19 items selected and the approximate objective value is 486.9426.
3.3. Approximation Algorithm of KP
Let describe the best approximate solution to KP in (26).
We can obtain the approximate solution of KP by Algorithm 2.

In Example 1, we obtain when k = 19.
The best approximate solution for KP and its objective value are as follows:
It may be proved that is the optimal solution to KP in Example 1.
3.4. Calculation Analysis
Approximation algorithms for KP based on upper bound of EkKP runs in polynomial time. Calculation for the upper bound of EkKP is less than O (nlnn) for a given EkKP. We obtain the approximate solution to KP on the basis of the upper bound of EkKP where k is an integer close to s. For a given k, the initial solution is determined by (13). It takes O (nlnn) time to develop the initial solution in the neighborhood (h = 1, 2, …, 4) defined by (21). Hence, it takes at most O (nlnn) time to achieve the approximate solution to EkKP. Generally, we obtain the best approximate solution for KP among the approximate solutions for EkKP where k is near the critical item s. Hence, the calculation for approximate solution to KP is less than O (nlnn).
In Algorithm 1, we search for the best solution in defined by (21). In order to develop the approximate solution quickly, the weight sum of h items is sorted in ascending order firstly, and then the profit sum of h items selected in the knapsack is compared with the profit sum of h items not selected. So the storage needed in Algorithm 1 is O(n) when h is fixed at 1, 2, 3, or 4. In Algorithm 2, we have the best solution to KP by comparison to the best approximate solution to EkKP where k is an integer close to s. So the storage of the approximate algorithm for KP is O(n).
3.5. Accuracy Analysis
All feasible solutions to KP are in the search scope of approximation algorithm with changing k and h. So approximation algorithm for KP is an εapproximation algorithm. From one aspect, weights, profits, capacity and size of KP have influence on the accuracy of an approximation algorithm. From another aspect, the search scope of the approximation algorithm has influence on its accuracy as well. Better solution is usually with more calculation. The exact algorithm for KP may be explored on the basis of the branch and bound algorithm here. Intensive research will be carried out in the future.
4. Computational Experiments
By Algorithm 2, we get the approximate solutions to KP listed in Table 5. The results are listed in Table 6. The optimal value listed in Table 6 is carried out by combo [24].
From Table 6, we find that the approximate solutions are almost the optimal solutions in 18 instances. It implicates that the approximate algorithm proposed here achieves high precision in solving KP.
In Table 7 lists the experimental results of the solutions for KP listed in Table 2 by Algorithm 2. The upper bound listed in Table 4 is carried out by equation (11) in Section 2.
From Table 7, we see that the relative average error of 100 instances is almost less than 0.0001. The experiment result shows that the approximate algorithm proposed here can achieve high accuracy.
From Table 8, we find that the upper bound of EkKP makes key contribution in FPTAS. Firstly, the initial solution constructed on the basis of the solution to the upper bound of EkKP is similar with the optimal solution. For example, there are only 4 elements different between the initial solution and the optimal solution to EkKP where k equals to 19, while there are 8 elements different between the initial solution constructed by efficiency and the optimal solution to KP in Table 8. Secondly, the differences between the initial solution and the solution to the upper bound of EkKP are near to the item . It is to say, we search the optimal solution to KP in a core with small size on the basis of the solution to the upper bound of EkKP. While the optimal solution to KP in a core with large size on the basis of the solution to the upper bound of Dantzig. So, algorithms proposed here is easy to get approximate solution to KP.
Furthermore, no better solution of EkKP exists when the objective value f_{a} of the approximate solution achieved before is larger than the upper bound B_{k}. So we only search solution of EkKP with upper bound larger than f_{a} which is in the neighborhood of the optimal solution to the upper bound of EkKP. The candidate strategy makes the search in polynomial time and the solution with high accuracy in Algorithm 1. The upper bound of EkKP plays important role in decreasing calculation of the approximate solution of EkKP in Algorithm 2 as well.
5. Conclusion
It is still difficult to obtain the exact solution for large scale 01 knapsack problem directly. Here a fast polynomial approximate solution is proposed on the basis of the upper bound for KP. The exact solution to KP is in the neighborhood of the solution to the upper bound for EkKP. Therefore, it is possible to find an approximation with high accuracy in the neighborhood of the solution to the upper bound for EkKP where k is near to the critical item s. All in all, as the basis of fast exact algorithm for KP, it is important to obtain an approximate solution and the upper bound for KP. In order to obtain a fast exact solution to KP, more intensive research on variables reduction need be conducted in the future.
Data Availability
All data inside the manuscript have been specified clearly in the manuscript.
Conflicts of Interest
The authors declare that they have no known conflicts financial interests or personal relationships that could have appeared to influence the work reported in this paper.