The multiple knapsack problem (MKP) is defined as follows. We are given a set of items and a set of bins (knapsacks) such that the -th bin has a capacity and an item has a size and a profit . The task is to select a subset of items such that the sum of their profits is maximized and they can be packed into those bins without exceeding the capacities. It is well-known that the problem admits a PTAS but has no FPTAS assuming P NP [17, 7, 16].
In this paper, we generalize the above problem to the submodular multiple knapsack problem (SMKP), where, instead of specifying a profit for each item, the profit of each subset of items is described by a monotone submodular function defined over all subsets of . The task is again to select a subset of items which maximizes the value of and can be packed into those bins without exceeding the capacities.
By introducing a monotone submodular objective function, SMKP falls into the field of submodular function maximization, which enjoys a long history of more than forty years. As early as in 1978, a simple greedy algorithm was proposed which yields a -approximation for the problem of maximizing a monotone submodular function subject to a cardinality constraint , and a approximation for the problem of maximizing a monotone submodular function subject to a matroid constraint . On the other hand, even for the cardinality constraint, submodular maximization problems can not be approximated within a ratio better than . Since then, it remains a central open question in this field whether or not the problem of maximizing a monotone submodular function admits a -approximation when subject to a matroid constraint. Until 2008, a groundbreaking work  answers this question affirmatively by proposing the so-called continuous greedy technique. Since then, a plenty of tight or well-performed approximation algorithms have been devised to maximize (monotone or non-monotone) submodular functions over a variety of constraints [2, 4, 5, 9, 12, 13, 15, 20, 21, 26]. Among them, the most relevant constraint to this work is the -knapsack constraint.
The -knapsack constraint assumes that there are knapsacks and the -th knapsack has budget . In contrast to our problem, the constraint assumes that each picked element incurs a cost in the -th knapsack for all . a set of elements is feasible if it is feasible in each knapsack. In some sense, under the -knapsack constraint, each picked element is packed into all knapsacks simultaneously; while in our model, under the capacity constraint, each picked element is packed into only one knapsack (we call it bin instead of knapsack in the paper). The -knapsack constraint is closely related to the capacity constraint in our problem. Firstly, the submodular maximization problem subject to the -knapsack constraint (often known as a knapsack constraint) generalizes the classical knapsack problem, and is exactly SMKP with a single bin as input. Secondly, SMKP can actually be reduced to the submodular maximization problem subject to the -knapsack constraint. The reduction is as follows. For an instance of SMKP, an item can be divided into items and for , and incurs a cost in the -th bin and a zero cost in other bins. The function value of a set of new items is equal to the value of the set of the original items from which they are generated. And now we have obtained a submodular maximization instance subject to the -knapsack constraint. Actually, SMKP is strictly simpler than the submodular maximization problem subject to the -knapsack constraint. The latter problem is NP-hard to approximate within an factor when is a part of input, while it is not difficult to find a constant approximation algorithm for SMKP for arbitrary (see the following subsection for details). However, finding a tight approximation algorithm for SMKP is a highly non-trivial task. Such tight algorithms have been the central topic in the field of submodular function maximization.
1.1 Our Results and Techniques
In this work we answer the aforementioned question affirmatively by presenting tight algorithms for SMKP in most cases: 1) constant case where the number of bins is a constant; 2) identical-size case where all the bins have the same capacity ; 3) bounded-size case where the ratio between the maximum capacity and the minimum capacity of the bins is bounded by a constant. The following table summarizes our results in this paper and lists the best known results previously as comparison.
|Case||Previous result||Our result|
|(randomized, continuous greedy)||(deterministic)|
|(randomized, continuous greedy)|
|(randomized, continuous greedy)|
The main contribution in this paper is the tight -approximation algorithm in the identical-size case and the approximation algorithm in the bounded-size case. Both algorithms are deterministic and based on the greedy technique with several new insights in the analysis. In the study of submodular maximization under variant constraints, though most state-of-art ratio is achieved by randomized algorithm, the simple deterministic algorithm always takes an important place. However, only very few problems have tight deterministic algorithms now: monotone submodular maximization under cardinality constraint , monotone submodular maximization under one knapsack constraint [18, 24], and unconstrained non-monotone submodular maximization . Our results contain new deterministic algorithms on submodular maximization and might inspire other results in the area.
All the algorithms in this paper are built on the greedy algorithm presented in Section 2.2. The solution returned by the greedy algorithm enjoys a good approximation. Formally, for any set ,
Here is the size of . On the other hand, is infeasible in that the capacity of each bin is violated by the last element added into it. These elements are referred to as reserved elements. By plugging into the last inequality and combining the fact that total capacity , we obtain an infeasible solution which admits -approximation. The algorithms in this paper are all about how one manages to round the greedy solution into a feasible one by dealing with the reserved elements carefully, such that the rounded solution will not lose too much in the approximation ratio.
One widely used technique to handle this problem is the enumeration technique. By enumerating all possible partial solutions, we are able to pack large-valued elements in . Then we can fill the bins by greedily picking small-valued elements, rendering the reserved elements to be small-valued. This idea provides us a simple polynomial time approximation algorithm when the number of bins is constant.
There is a deterministic combinatorial algorithm for SMKP which admits a -approximation and runs in time, specially when the number of bins is a constant, the algorithm runs in polynomial time.
Remark. When the number of bins is a constant, the randomized algorithm for the -knapsack constraint  already achieves a -approximation for SMKP. However, such algorithm adopts the continuous greedy technique and requires an involved rounding procedure.
When applying such idea to arbitrary , the obstacle is that the enumeration step requires exponential time. This is because the “constant-bins” algorithm takes all the bins into consideration simultaneously. A natural idea to resolve it is to pack bins one at a time. For each bin, we manage to pack elements of as large marginal values as possible. This is actually running a greedy algorithm on an exponential size submodular maximization problem (see the reductions in Section 2.1). As a result, to achieve a -approximation, we find it suffices, taking the first bin as an example, to find a feasible set with and , where is the optimal solution of the SMKP instance. This is possible by observing that if one packs the first bin by the greedy algorithm, the returned infeasible set satisfies
The ratio is approximately when is small enough. For the identical-size case and the bounded-size case, this is exactly the case when is large.
Our task is again to handle the reserved element in carefully. One thus may expect the enumeration technique still works here. However, this is not the case. The reason is simple: once the bin has packed some elements before the greedy process, the remaining capacity of the bin would become smaller, and therefore no longer a desired fraction of .
In order to tackle such difficulty, the first trick we use is to enumerate large-sized elements rather than large-valued ones, and to greedily pick small-sized elements rather than small-valued ones. There are several advantages to classify elements according to their sizes instead of values. Firstly, the value function is submodular while the size function is linear. Thus we do not need to handle the difficulties coming from the change of marginal values. Such division is more stable and will not change in the whole process. Secondly, dividing elements according to their sizes allows us to manipulate them in a more flexible way, e.g. small-sized elements can be transferred among bins. Thirdly, in the analysis of greedy subroutine, we have. Since the ratio mainly depends on the ratio between capacities, the based-on-size division is also useful in the analysis. In the following, we further explain how to achieve the desired ratio in the identical-size case and the bounded-size case, respectively.
For the identical-size case, the basic framework of the algorithm is as follows. For each bin, we adapt the enumeration step by only enumerating all feasible solutions of large-sized elements. Then we fill the remaining part of the bin for each enumerated set by packing only small-sized elements greedily. Finally, the set of the maximum value is returned.
Take the first bin as an example. Let and be the set of large-sized and small-sized elements in , respectively. Let be the set packed into the first bin and let and be the set returned by the enumeration step and the greedy step, respectively. To prove , we can show that and . The first inequality is easy to achieve by the enumeration process, while the second one is satisfied if we have . Equivalently, we hope the algorithm can find a suitable set such that
However, this is obviously a far-fetched condition for . To assist the analysis, our first key observation is that under mild precondition, we can still prove once the following condition holds for :
To summarize, we find two distinct sufficient conditions for finding a good . Unfortunately, we find it impossible to achieve either conditions by considering only a single bin, even if we only require that they are satisfied approximately. One can imagine that the elements in might be packed into the bins by in an unbalanced manner. That is, some bins contain elements in with very large values as well as very large sizes, while other bins contain elements in with small values as well as small sizes.
In order to circumvent this difficulty, we introduce the average argument. Let be its partition into the identical bins. Intuitively, we want to group constant sets together such that the average size of each group is always on the one side of . That is, either all of them are smaller than , or they are larger than . This can be approximately achieved by a coupling process: order the sets by their sizes and group them pairwise such that the largest one is coupled with the smallest one, the second to largest one is coupled with the second to smallest one, etc. After constant rounds of this process, the returned groups will satisfy the desired property. More specifically, let be the returned groups. Let denote the number of set in , which means can be packed into bins. Then, we have a) , or b) . In either case, we can find a set which approximately satisfies one of the aforementioned conditions for a good , with a compromise that needs to be packed into multiple bins. This suggests that the overall algorithm needs to pack multiple (but constant) bins one at a time.
Combining all these techniques, we have the following result for the identical-size case of SMKP.
For the identical-size case of SMKP, there is a polynomial time deterministic algorithm which yields a -approximation.
Next, we manage to generalize the above result to the bounded-size case where the ratio between the maximum capacity and the minimum capacity of the bins is bounded by a constant . Clearly, the identical-size case is a special case of the bounded-size case by setting . The algorithm framework is quite similar as the identical-size case, but the analysis uses different ideas. The algorithm pack bins one at a time. For each bin, the algorithm enumerate large-sized elements, greedily pick small-sized elements, and pack the reserved element into the reserved bins. The fact that is a constant ensures that the algorithm is polynomial.
For the analysis, we combine the idea of classifying elements based on their sizes and the analysis for the classical greedy algorithm under a matroid constraint . Let denote the elements packed in the -th bin and denote the elements packed in the first bins by an algorithm. In a standard analysis, if one can find a partition of such that , then the algorithm admits a approximation. We show that we can actually construct such a partition for our algorithm. The key observation is that the small elements in can be arbitrarily re-partitioned according to our requirements, since small-sized elements can be transferred freely among the bins. To conclude, we have the following result.
For the bounded-size case of SMKP, there is a polynomial time deterministic algorithm which yields a -approximation.
The ratio coincides with the ratio of the classical greedy algorithm for a matroid constraint and therefore is acceptable (the state of art deterministic algorithm  achieves a approximation for a matroid constraint). It is also expected that a -approximation can be achieved by applying the continuous greedy technique. However, the standard implementation  can only achieve a approximation. Instead, we adopt a modified continuous greedy process in 
to interpolate ouralgorithm with the continuous greedy process, which results in a tight approximation ratio successfully.
For the bounded-size case of SMKP, there is a polynomial time randomized algorithm which yields a -approximation.
1.2 Related Work
The multiple knapsack problem has been fully studied previously. Kellerer  proposed the first PTAS for the identical-size case of the problem. Soon after, Chekuri and Khanna  proposed a PTAS for the general case. The result was later improved to an EPTAS by Jansen . On the other hand, it is easy to see that the problem does not admit an FPTAS even for the case of bins unless P NP, by reducing the partition problem to it. For the problem of maximizing a monotone submodular function subject to a knapsack constraint, a tight algorithm was known which runs in time [17, 24]. Later, a fast algorithm was proposed in  which achieves a -approximation and runs in time. This was recently improved in  by a new algorithm which runs in time. For the problem of maximizing a monotone submodular function subject to the -knapsack constraint, there is a tight algorithm  when the number of knapsacks is a constant. The problem is NP-hard to approximate within an factor when is a part of input, since it contains the Independent-set problem as a special case. For the submodular multiple knapsack problem considered in this paper, to the best of our knowledge, nobody except Moran Feldman has studied this problem. In his Ph. D thesis , Feldman proposed a polynomial time -approximation algorithm and a pseudo polynomial time approximation algorithm for the problem. For the identical-size case, he improved the results to a polynomial time -approximation algorithm and a pseudo polynomial time -approximation algorithm. We note that by generalizing the algorithms for the multiple knapsack problem in , there is a deterministic algorithm for the identical-size case and a deterministic algorithm for the general case. The latter ratio can be improved to by applying the approximate continuous greedy process in 
We first present a formal description of the submodular multiple knapsack problem (SMKP) and some notations in Section 2. The greedy algorithm is also presented in this section. Then we present a tight algorithm for SMKP in Section 3, assuming the number of bins is a constant. Tight algorithms for the identical-size case and the bounded-size case of SMKP are presented in Section 4 and Section 5, respectively. Algorithms from these two sections assume that the number of bins is “large”. Finally, we conclude the paper in Section 6.
Let be a set function. is monotone if for any . is submodular if for any and . Here is a shorthand for . Throughout this paper, we also use and to denote the marginal gains and , respectively. An explicit representation of requires an exponential space with respect to the number of elements . Thus we assume that is accessed by a value oracle that given a set , returns .
An instance of the submodular multiple knapsack problem (SMKP) is defined as follows. We are given a ground set of elements and bins (knapsacks). For , the -th bin has a positive capacity . Each element has a positive size , and the size of a set is . In addition, there is a non-negative objective function defined over all subsets of . In this paper, the objective function is assumed to be monotone and submodular. A set is feasible if there exists an ordered partition of such that for each . In other words, is feasible if there is a way to pack it into those bins. The way how is packed does not change the value of . The goal is to find a feasible set (as well as the way it is packed) which maximizes the value of objective function .
For SMKP, the identical-size case refers to the case where all bins have the same capacity . The bounded-size case refers to the case where where the ratio between the maximum capacity and the minimum capacity of the bins is bounded by a constant . Throughout the paper, when we refer to a set , we assume that some of its partition is implicitly given. The partition may be generated from an algorithm, or be specified explicitly if necessary. Depending on the context, sometimes the partition might not be a feasible solution for the problem, that is, might not hold for some . We use to denote the number of elements in and to denote the number of bins used in its partition . The term actually depends on the partition of , but we slightly abuse the notation here for ease of representation.
The multilinear extension.
For a set function , its multilinear extension is defined as , where is a random set such that
appears in it independently with probability. Given , can be approximately computed within an arbitrary small multiplicative error by the Monte Carlo sampling. For ease of representation, we assume in this paper there is an oracle which returns the exact value of when given .
2.1 An Exponential Size Reduction
In the following two theorems, we show that an instance of SMKP can be reduced to an exponential-size instance of maximizing a monotone submodular function subject to a matroid constraint. This fact help understand how our algorithms perform and why they can achieve the desired ratios. It is also used in the continuous greedy algorithm for the bounded-size case. We note that the reduction is similar to that for generalized assignment problem in .
An instance of SMKP can be reduced to an exponential-size instance of maximizing a monotone submodular function subject to a partition matroid.
Given an instance of SMKP, we construct the reduction as follows. Let for and . Here and from distinct and may contain the same elements. However, we still regard them as two distinct sets, distinguished by their indexes. Then the capacity constraint can be translated into a partition matroid constraint which requires to find a set such that for . We then define a function over all subsets of such that for a set , . Clearly, is monotone, since does. Besides, for and , . Thus is also submodular. This completes the proof. ∎
An instance of SMKP in the identical-size case can be reduced to an exponential-size instance of maximizing a monotone submodular function subject to a cardinality constraint.
Given an instance of SMKP, we construct the reduction as follows. Let . Then the capacity constraint can be translated into a cardinality constraint which requires to find a set such that . The definition of the objective function is the same as that in the previous theorem. ∎
2.2 The Greedy Algorithm
The greedy algorithm is shown as Algorithm 1. It returns a (possibly infeasible) set that admits a -approximation. All the algorithms in this paper are built based on it. It selects elements to be packed in a greedy manner, according to the densities of elements’ current marginal values to their sizes. It packs the selected elements into bins as long as there is a bin whose capacity has not been fully used. As a result, Algorithm 1 might not return a feasible solution. However, it is easy to see that each bin packs at most one additional element. That is, if for some and is the last element added into , then and hence is feasible in the -th bin. For convenience, we say Algorithm 1 returns an almost feasible solution. For each bin, the last element that violates the capacity constraint is called a reserved element. The following lemma lower bounds the quality of the almost feasible set returned by Algorithm 1.
Let be the set returned by Algorithm 1. For any set , we have
We assume that , since otherwise all elements in have been packed into and therefore the lemma holds trivially. Assume that , and for , denotes the first elements picked by the algorithm. Then, by the greedy rule,
This gives us
The first inequality holds since is submodular. The second inequality holds since is monotone and .
We also assume that , since otherwise the lemma already holds. Under this assumption, it must holds that , since otherwise the inequality
implies that . A contradiction!
By rearranging the last inequality, we have
The recurrence gives us
The second inequality holds due to . Hence we have
The lemma follows immediately from it. ∎
Let be an optimal solution. The above lemma immediately leads to the following corollary.
The set returned by Algorithm 1 satisfies .
3 Constant Number of Bins
In this section, we present a tight algorithm for SMKP, assuming the number of bins is a constant. If we use the greedy algorithm in Section 2.2 to pack elements into those bins, we are able to obtain a set with the approximation ratio , but this set might be infeasible, with a reserved element in each bin. If the reserved elements are of small values, we can just discard these elements without losing too much. However, the greedy algorithm itself cannot guarantee this property. In light of this, Algorithm 2 manages to first pack elements of large values in an optimal solution by the enumeration technique, and then pack elements of small value by the greedy algorithm. In doing so, it ensures that the values of the reserved elements are small and therefore can be safely discarded.
Algorithm 2 admits a -approximation and runs in time. When is a constant, it runs in polynomial time.
Denote by the optimal solution. Assume w.l.o.g. that , since otherwise will be enumerated in the enumeration step. We order elements in greedily according to their marginal values, i.e. , , etc. In the enumeration step, the solution must be visited such that contains exactly the first elements in and these elements are packed in the same way as in . In the following analysis, we focus on this solution and show that achieves the desired ratio. Since the algorithm returns the solution with the maximum value, this completes the proof.
We claim that for any . Let be the first elements in . Then for and any , . Summing up from to , we have . By plugging , . The claim holds. This implies that . As a result, elements in will not be excluded from the execution of the greedy algorithm. Besides, since elements in are packed in the same way as in , is actually a feasible (indeed optimal) solution when invoking the greedy algorithm. By Corollary 1, the set returned by Greedy satisfies
Since is obtained from by discarding at most reserved elements in , by the submodularity of ,
Hence we have
The last inequality holds since .
4 The Identical-size Case
In this section, we present a tight deterministic algorithm for the identical-size case of SMKP where all bins have the same capacity . The main algorithm is depicted as Algorithm 3. Given a constant as input, it requires that the number of bins and will achieve a approximation. When , we can use Algorithm 2 to achieve a tight -approximation. Combining these two results, we resolve the identical-size case completely and Theorem 2 follows.
In Algorithm 3, the first bins are called working bins. In contrast, the last bins are called reserved bins. It mainly uses working bins to pack elements. A bin is called empty if no elements have been packed in this bin. Whenever there remains at least empty working bins, Algorithm 3 attempts to pack empty working bins with the remaining elements by invoking Algorithm 4. Let be the set returned by an execution of Algorithm 4 and denote the number of bins that Algorithm 4 used to pack . We will show in Lemma 2 that the marginal value of w.r.t. the elements already packed is approximately a fraction of the margin value of . This directly leads to the desired ratio (Theorem 8).
Algorithm 4 divides elements into two classes according to their sizes. Given input , an element is large if and small otherwise. Let be the set of large elements in and be the set of small elements. From a high-leveled viewpoint, Algorithm 4 first enumerates all feasible ways of packing only the large elements. Then for each enumerated solution, small elements are packed into it by invoking the greedy algorithm. Finally, the one with the maximum “average value” will be returned.
Note that if an enumerated solution only uses bins, Algorithm 4 will only add small elements to those non-empty bins. The only exception is that when (or equivalently ), Algorithm 4 will use a single bin to pack small elements. As a result, although Algorithm 3 invokes Algorithm 4 with bins, the set returned by Algorithm 4 might use fewer than bins. Besides, the solution might not be feasible. Due to the greedy algorithm, might contain a reserved element in each of the bins it uses. Nonetheless, we still let Algorithm 4 returns an infeasible solution and tackle the reserved elements in Algorithm 3. At this point, the reserved bins come to rescue. Observe that all reserved elements must be small elements. Thus the empty reserved bins can pack at least reserved elements. On the other hand, there are only working bins; each of them contains at most one reserved element. Consequently, all reserved elements can be packed into the reserved bins without exceeding the capacities, thus, line 4 in Algorithm 3 can always be executed.
We remark that though Algorithm 2 and Algorithm 4 are very similar, there are some differences between them. First, in the enumeration step, Algorithm 2 enumerates feasible solutions containing constant number of elements, while Algorithm 4 enumerates feasible solutions of large elements. Second, in the greedy step, Algorithm 2 directly discards the reserved elements, while Algorithm 4 retains the reserved elements and may return an infeasible solution.
We now introduce some notations for sake of analysis. Assume that the while loop in Algorithm 3 has been executed times. For , denote by the set returned by Algorithm 4 in the -th loop and by the packed elements after the -th loop. Then . Recall that is the number of bins used to pack by Algorithm 4. Note that does not count the number of reserved bins used to pack the reserved elements in . Let be the optimal solution. The following lemma bounds the marginal value of with respect to in terms of the marginal value of .
For , we have
Algorithm 3 admits a -ratio and runs in time.
By Lemma 2 and the monotonicity of , we have
By rearranging the above inequality, we obtain
This gives us
On the other hand, by the ending condition of the while loop,
The last inequality holds since . Combining the above inequalities, we have
which implies that
The running time follows by observing that the algorithm runs in at most rounds, the enumeration step costs time and the greedy algorithm needs time. ∎
4.1 Proof of Lemma 2
This section is dedicated to prove Lemma 2. For simplicity, we show that the lemma holds for the first iteration of the while loop in Algorithm 3. A similar argument holds for all comping loops, in which it suffices to replace a set by and the function value by for the -th loop. Let be the set of large elements in and be the set of small elements in . We prove Lemma 2 by a case analysis, based on the densities of the large and small elements in (Lemma 3 and Lemma 5).
If holds, then .
The second inequality holds since for . The third inequality holds since and . The second to last inequality holds due to the condition of the lemma and the monotonicity of . The last inequality holds as long as . Thus in this case the lemma holds.
If , again by Lemma 1,
Thus in this case the lemma holds.
Below we assume that and . Combining these two assumptions, we then show the small elements in only contributes a negligible fraction of the value function:
The first inequality holds since is monotone increasing. The second inequality holds since for . The third inequality holds as long as . Hence by the submodularity of ,
Let . Then in the enumeration step of Algorithm 4, the feasible solution where a single bin is used to pack will be enumerated, and it holds that
Thus in this case the lemma holds. This completes the proof. ∎
If the condition of Lemma 3 does not hold, by the submodularity of , it must hold that