Maximizing Submodular or Monotone Functions under Partition Matroid Constraints by Multi-objective Evolutionary Algorithms

06/23/2020
by   Anh Viet Do, et al.
0

Many important problems can be regarded as maximizing submodular functions under some constraints. A simple multi-objective evolutionary algorithm called GSEMO has been shown to achieve good approximation for submodular functions efficiently. While there have been many studies on the subject, most of existing run-time analyses for GSEMO assume a single cardinality constraint. In this work, we extend the theoretical results to partition matroid constraints which generalize cardinality constraints, and show that GSEMO can generally guarantee good approximation performance within polynomial expected run time. Furthermore, we conducted experimental comparison against a baseline GREEDY algorithm in maximizing undirected graph cuts on random graphs, under various partition matroid constraints. The results show GSEMO tends to outperform GREEDY in quadratic run time.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/20/2020

Optimising Monotone Chance-Constrained Submodular Functions Using Evolutionary Multi-Objective Algorithms

Many real-world optimisation problems can be stated in terms of submodul...
04/20/2021

Multi-objective Evolutionary Algorithms are Generally Good: Maximizing Monotone Submodular Functions over Sequences

Evolutionary algorithms (EAs) are general-purpose optimization algorithm...
10/12/2019

Multi-objective Evolutionary Algorithms are Still Good: Maximizing Monotone Approximately Submodular Minus Modular Functions

As evolutionary algorithms (EAs) are general-purpose optimization algori...
12/16/2020

Pareto Optimization for Subset Selection with Dynamic Partition Matroid Constraints

In this study, we consider the subset selection problems with submodular...
11/20/2017

Maximizing Non-monotone/Non-submodular Functions by Multi-objective Evolutionary Algorithms

Evolutionary algorithms (EAs) are a kind of nature-inspired general-purp...
01/21/2021

GPU-Accelerated Optimizer-Aware Evaluation of Submodular Exemplar Clustering

The optimization of submodular functions constitutes a viable way to per...
07/13/2021

A Parallel Approximation Algorithm for Maximizing Submodular b-Matching

We design new serial and parallel approximation algorithms for computing...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The area of runtime analysis has made significant contributions to the theory of evolutionary algorithms over the last 25 years [1, 20]

. Important results have been obtained for a wide range of benchmark functions as well as for important combinatorial optimization problems 

[33]

. This includes a wide range of evolutionary computing methods in a wide range of deterministic, stochastic and dynamic settings. We refer the reader to 

[12] for a presentation of important recent research results.

Many important real-world problems can be stated in terms of optimizing a submodular function and the analysis of evolutionary algorithms using multi-objective formulations has shown that they obtain in many cases the best possible performance guarantee (unless P=NP). Important recent results on the use of evolutionary algorithms for submodular optimization are summarized in [42]. The goal of this paper is to expand the investigations of evolutionary multi-objective optimization for submodular optimization. While previous investigations mainly concentrated on monotone submodular functions with a single constraint, we consider non-monotone submodular functions with a set of constraints.

1.1 Related work

Submodular functions are considered the discrete counterparts of convex functions [31]. Submodularity captures the notion of diminishing marginal return, and is present in many important problems. While minimizing submodular functions can be done using a polynomial time combinatorial algorithm [19], submodular maximization encompasses many NP-hard combinatorial problems such as maximum coverage, maximum cut [18], maximum influence [22], and sensor placement problem [24, 23]

. It is also applied in many problems in machine learning domain

[30, 41, 29, 28, 38]. Considering the role of evolutionary algorithms in difficult optimization problems, we focus on submodular maximization.

Realistic optimization problems often impose constraints on the solutions. In applications of submodular maximization, Matroid and Knapsack constraints are among the most common [26]. In this work, we consider submodular maximization under partition matroid constraints, which are a generalization of cardinality constraints. This type of constraint has been considered in a variety of applications [21, 7, 14].

A greedy algorithm has been shown to achieve -approximation ratio in maximizing monotone submodular functions under partition matroid constraints [10]. It was later proven that is the best approximation ratio a polynomial time algorithm can guarantee. A more recent study [6] proposed a randomized algorithm that achieves this ratio. Another study [5]

analyzes derandomizing search heuristics, leading to a deterministic

-approximation ratio.

Additionally, more nuanced results have been reached when limiting objective functions to those with finite rate of marginal change, quantified by curvature as defined in [15]. The results in [8, 40] indicate that -approximation ratio is achievable by the continuous greedy algorithm in maximizing monotone submodular functions under a matroid constraint. A more recent study [4] proved -approximation ratio for the deterministic greedy algorithm in maximizing functions with submodularity ratio , under a cardinality constraint.

These results rely on the assumption of monotonicity of the objective functions, for all , which do not hold in many applications of submodular maximization. A study [17] derives approximation guarantees for GSEMO algorithm in maximizing monotone and symmetric submodular function under a matroid constraint, which suggest that non-monotone functions are harder to maximize. This is supported by another result [15] for a greedy algorithm in maximizing general submodular function under partition matroid constraints. A recent study [36] extends the results for a GSEMO variant to the problems of maximizing general submodular functions, but under a cardinality constraint.

1.2 Our contribution

In this work, we contribute to the theoretical analysis of GSEMO by generalizing previous results to partition matroid constraints. Firstly, we provide an approximation guarantee for GSEMO in maximizing general submodular functions under partition matroid constraints (Theorem 1). Secondly, we derive another result for monotone and not necessarily submodular functions, under the same constraints (Theorem 2), to account for other important types of function like subadditive functions. Subadditivity encompasses submodularity, and is defined by the property where the whole is no greater than the sum of parts. Subadditive functions are commonly used to model items evaluations and social welfare in combinatorial auctions [2, 3, 39, 15]. Our results extend the existing ones [36] with more refined bounds.

We investigate GSEMO’s performance against GREEDY’s [15] in maximizing undirected cuts in random graphs under varying cardinality constraints and partition matroid constraints. Graph cut functions with respect to vertices sets are known to be submodular and non-monotone [13]. In particular, they are also symmetric for undirected graphs [37]. Our results suggest that GSEMO typically requires more evaluations to reach GREEDY’s outputs quality. Nonetheless, GSEMO surpasses GREEDY shortly after the latter stops improving, indicating the former’s capacity for exploring the search spaces. Predictably, GSEMO outperforms GREEDY more reliably in larger search spaces.

The paper is structured as follows. We formally define the problems and the algorithms in Section 2. In Section 3, we analyze GSEMO with respect to its approximation behaviour and runtime and report on our experimental investigations in Section 4. Finally, we finish with some conclusions.

2 Preliminaries

In this section, we provide a formal definition of the problem and some of its parameters relevant to our analyses. We also describe the simple GREEDY algorithm and the GSEMO algorithm considered in this work.

2.1 Problem definition

We consider the following problem. Let be a non-negative function over a set of size , be a partition of for some , be integers such that for all , the problem is finding maximizing , subject to

These constraints are referred to as partition matroid constraints, which are equivalent to a cardinality constraint if . The objective function of interest is submodular, meaning it satisfies the property as defined in [32]

Definition 1.

A function is submodular if

We can assume that is not non-increasing, and for monotone , we can assume . To perform analysis, we define the function’s monotonicity approximation term similar to [24], but only for subsets of a certain size

Definition 2.

For a function , its monotonicity approximation term with respect to a parameter is

for and .

It is clear that is non-negative, non-decreasing with increasing , and is monotone iff . Additionally, for monotone non-submodular , we use submodularity ratio which quantifies how close is to being modular. In particular, we simplify the definition [11] which measures the severity of the diminishing return effect.

Definition 3.

For a monotone function , its submodularity ratio with respect to two parameters , is

for and .

It can be seen that is non-negative, non-increasing with increasing and , and is submodular iff for all .

For the purpose of analysis, we denote , , and the optimal solution; we have and . We evaluate the algorithm’s performance via where is the algorithm’s output. Furthermore, we use the black-box oracle model to evaluate run time, hence our results are based on numbers of oracle calls.

2.2 Algorithms descriptions

A popular baseline method to solve hard problems is greedy heuristics. A simple deterministic GREEDY variant has been studied for this problem [15]. It starts with the empty solution, and in each iteration adds the feasible remaining element in that increases value the most. It terminates when there is no remaining feasible elements that yield positive gains. This algorithm extends the GREEDY algorithms in [32] to partition matroids constraints. Note that at iteration , GREEDY calls the oracle times, so its run time is . According to [15], it achieves approximation ratio when is submodular, and approximation ratio when is non-decreasing subadditive, with being the curvature of .

On the other hand, GSEMO [25, 16, 17], also known as POMC [35], is a well-known simple Pareto optimization approach for constrained single-objective optimization problems. It has been shown to outperform the generalized greedy algorithm in overcoming local optima [35]. To use GSEMO with partition matroid constraints, the problem is reformulated as a bi-objective problem

where , .

  Input: a problem instance:
  Parameter: the number of iterations
  Output: a feasible solution
  ,
  while  do
     Randomly sample a solution from
     Generate by flipping each bit of

independently with probability

     if  then
        
     end if
  end while
  return
Algorithm 1 GSEMO algorithm

GSEMO optimizes two objectives simultaneously, using the dominance relation between solutions, which is common in Pareto optimization approaches. By definition, solution dominates () iff and . The dominance relation is strict () iff or . Intuitively, dominance relation formalizes the notion of “better” solution in multi-objective contexts. Solutions that don’t dominate any other present a trade-off between objectives to be optimized.

The second objective in GSEMO is typically formulated to promote solutions that are “further” from being infeasible. The intuition is that for those solutions, there is more room for feasible modification, thus having more potential of becoming very good solutions. For the problem of interest, one way of measuring “distance to infeasibility” for some solution is counting the number of elements in that can be added to before it is infeasible. The value then would be , which is the same as in practice. Another way is counting the minimum number of elements in that need to be added to before it is infeasible. The value would then be . The former approach is chosen for simplicity and viability under weaker assumption about the oracle.

On the other hand, the first objective aims to present the canonical evolutionary pressure based on objective values. Additionally, also discourages all infeasible solutions, which is different from the formulation in [35] that allows some degree of infeasibility. This is because for , there can be some infeasible solution where . If is very high, it can dominate many good feasible solutions, and may prevent acceptance of global optimal solutions into the population. Furthermore, restricting to only feasible solutions decreases the maximum population size, which can improve convergence performance. It is clear the the population size is at most . These formulations of the two objective functions are identical to the ones in [17] when .

In practice, set solutions are represented in GSEMO as binary sequences, where with the following bijective mapping is implicitly assumed

This representation of set is useful in evolutionary algorithms since genetic bit operators are compatible. GSEMO operates on the bit sequences, and the fitness function is effectively a pseudo-Boolean function . It starts with initial population of a single empty solution. In each iteration, a new solution is generated by random parent selection and bit flip mutation. Then the elitist survival selection mechanism removes dominated solutions from the population, effectively maintaining a set of known Pareto-optimal solutions. The algorithm terminates when the number of iteration reaches some predetermined limit. The procedure is described in Algorithm 1. We choose empty set as the initial solution, similar to [35] and different from [17], to simplify the analysis and stabilize theoretical performance. Note that GSEMO calls the oracle once per iteration to evaluate a new solution, so its run time is identical to the number of iterations.

3 Approximation guarantees

We derive an approximation guarantee for GSEMO on maximizing a general submodular function under partition matroid constraints. According to the analysis for GREEDY [15], we can assume there are “dummy” elements with zero marginal contribution. For all feasible solutions where , let be the feasible greedy addition to , we can derive the following result from Lemma 2 in [36], using .

Lemma 1 ([36]).

Let be a submodular function and be defined in Definition 2, for all feasible solutions such that

With this lemma, we can prove the following result where denotes the population at iteration .

Theorem 1.

For the problem of maximizing a submodular function under partition matroid constraints, GSEMO generates in expected run time a solution such that

Proof.

Let be a statement , and , it is clear that holds, so and is well-defined for all since the empty solution is never dominated.

Assuming at some , let such that holds. If is not dominated and removed from , then . Otherwise, there must be some such that and . This implies , so . Therefore, is never decreased as progresses. Let , Lemma 1 implies

The second inequality uses . The probability that GSEMO selects is at least , and the probability of generating by mutating is at least . Furthermore, holds as shown, so if . Since and are chosen arbitrarily, this means

Therefore, the Additive Drift Theorem [27] implies the expected number of iterations for to reach from is at most . When , must contain a feasible solution such that

Therefore, GSEMO generates such a solution in expected run time at most . ∎

In case of a single cardinality constraint (), this approximation guarantee is at least as tight as the one for GSEMO-C in [36]. If monotonicity of is further assumed, the result is equivalent to the one for GSEMO in [17]. Additionally, the presence of suggests that the non-monotonicity of does not necessarily worsen the approximation guarantee when negative marginal gains are absent from all GSEMO’s possible solutions (i.e. cannot decrease objective values by adding an element).

As an extension beyond submodularity instead of monotonicity, we provide another proof of the approximation guarantee for GSEMO on the problems of maximizing monotone functions under the same constraints. Without loss of generality, we can assume that is normalized, meaning . We make use of the following inequality, derived from Lemma 1 in [34].

Lemma 2.

Let be a monotone function and be defined in Definition 3, for all feasible solutions such that

Using this lemma, we similarly prove the following result.

Theorem 2.

For the problem of maximizing a monotone function under partition matroid constraints, GSEMO with expected run time generates a solution such that

Proof.

Let be a statement , and , it is clear that holds, so and is well-defined for all since the empty solution is never dominated.

Assuming at some , there must be such that holds. If is not dominated and removed from , then . Otherwise, there must be some such that and . This implies , so . Therefore, is never decreased as progresses. Let , Lemma 2 implies

The second inequality uses .The probability that GSEMO selects is at least , and the probability of generating by mutating is at least . Furthermore, holds as shown, so . Since and are chosen arbitrarily, this means

Therefore, according to the Additive Drift Theorem [27], the expected number of iterations for to reach from is at most . When , must contain a feasible solution such that

Therefore, GSEMO generates such a solution in expected run time at most . ∎

Compared to the results in [35], it is reasonable to assume that restricting GSEMO’s population to only allow feasible solutions improves worst-case guarantees. However, it also eliminates the possibility of efficient improvement by modifying infeasible solutions that are very close to very good feasible ones. This might reduce its capacity to overcome local optima.

4 Experimental investigations

We compare GSEMO and GREEDY on the symmetric submodular Cut maximization problems with randomly generated graphs under varying settings. The experiments are separated into two groups: cardinality constraints () and general partition matroid constraints ().

4.1 Max Cut problems setup

Weighted graphs are generated for the experiments based on two parameters: number of vertices (which is ) and density. There are 3 values for : 50, 100, 200. There are 5 density values: 0.01, 0.02, 0.05, 0.1, 0.2. For each - pair, 30 different weighted graphs – each denoted as – are generated with the following procedure:

  1. Randomly sample from without replacement, until .

  2. Assign to a uniformly random value in for each .

  3. Assign for all .

Each graph is then paired with different sets of constraints, and each pairing constitutes a problem instance. This enables observations of changes in outputs on the same graphs under varying constraints. For cardinality constraints, , rounded to the nearest integer. Thus, there are 30 problem instances per -density- triplet. For partition matroid constraints, the numbers of partitions are . The partitions are of the same size, and each element is randomly assigned to a partition. The thresholds are all set to since the objective functions are symmetric. Likewise, there are 30 problem instances per -density- triplet.

GSEMO is run on each instance 30 times, and the minimum, mean and maximum results are denoted by GSEMO, GSEMO and GSEMO, respectively. The GREEDY algorithm is run until satisfying its stopping condition, while GSEMO is run for iterations. Their final achieved objective values are then recorded and analyzed. Note that the run time budget for GSEMO in every setting is smaller than the theoretical bound on average run time in Theorem 1, except for where it is only slightly larger.

4.2 Cut maximization under a cardinality constraint

width=center density GREEDY GSEMO GSEMO GSEMO L–W–T min max min max stat min max stat min max stat 50 0.01 13 6.638 12.71 6.625 12.59 - 6.635 12.7 - 6.638 12.71 * 3–3–24 25 6.638 13.27 6.625 13.13 - 6.65 13.25 * 6.706 13.27 + 2–7–21 38 6.638 13.27 6.625 13.13 - 6.647 13.24 * 6.706 13.27 + 2–6–22 0.02 13 12.16 18.08 12.1 18.08 - 12.15 18.08 * 12.17 18.08 + 6–5–19 25 13.5 20.27 13.43 20.17 - 13.47 20.29 + 13.5 20.33 + 8–16–6 38 13.5 20.27 13.39 20.23 - 13.47 20.3 + 13.5 20.33 + 7–17–6 0.05 13 22.09 29.38 21.69 29.28 - 22.03 29.49 + 22.09 29.51 + 2–17–11 25 26.52 37.14 27.18 36.28 - 27.75 36.95 + 28.1 37.14 + 4–20–6 38 26.52 37.14 27.39 36.43 - 27.89 37.02 + 28.2 37.14 + 5–21–4 0.1 13 38.64 47.14 38.19 46.82 - 38.59 47.12 * 38.69 47.14 + 5–11–14 25 46.61 57.93 45.36 57.38 - 46.85 57.85 + 47.28 58.03 + 11–16–3 38 46.61 58.08 45.03 57.58 - 46.77 57.95 + 47.28 58.1 + 10–17–3 0.2 13 63.13 77.38 63.09 77.01 * 63.26 77.43 + 63.46 77.64 + 2–16–12 25 78.89 92.57 79.37 91.68 - 80.61 92.25 + 80.82 92.66 + 5–21–4 38 78.89 92.57 79.82 91.78 - 80.62 92.31 + 80.82 92.66 + 5–21–4 100 0.01 25 24.93 31.88 24.9 31.86 - 25.11 31.89 * 25.36 31.89 + 7–9–14 50 27.87 37.79 27.56 37.87 - 28.11 38.07 * 28.67 38.26 + 12–13–5 75 27.87 37.79 27.24 37.71 - 28.07 38 + 28.67 38.26 + 11–13–6 0.02 25 42.95 53.66 42.95 53.56 - 43.38 53.64 * 43.4 53.66 + 7–10–13 50 51.93 66 51.95 65.27 - 52.62 66.14 + 52.78 66.4 + 9–15–6 75 51.93 66 51.69 64.31 - 52.6 66.08 + 52.8 66.4 + 9–17–4 0.05 25 78.13 94.98 78.09 95.3 - 78.38 95.45 + 78.78 95.49 + 8–17–5 50 100.6 120.7 100 119 - 100.9 120.3 + 101.7 120.7 + 7–18–5 75 100.7 120.7 99.23 118.9 - 100.8 120.4 + 101.9 120.7 + 7–17–6 0.1 25 138.9 155.3 138.5 155 - 139.1 156 + 139.6 156.2 + 2–20–8 50 178.4 197.8 177 198.3 * 178 199.9 + 179.6 200.6 + 7–17–6 75 178.4 197.8 176.6 198.6 * 178 199.9 + 179.4 200.6 + 6–17–7 0.2 25 224.1 249.2 222.8 249.4 * 224 250 + 225.6 250.2 + 4–22–4 50 297.6 325.1 297.8 323.9 * 300.9 325.8 + 302.7 326.4 + 6–20–4 75 297.6 325.1 298 323.9 - 300.4 325.8 + 303.2 326.4 + 6–19–5 200 0.01 50 85.54 96.11 84.98 96.05 * 85.58 96.3 + 85.84 96.52 + 7–20–3 100 103.1 118.4 103.5 118.5 - 104.6 120.1 + 104.9 121.4 + 4–23–3 150 103.1 118.4 103.6 118.6 - 104.7 120 + 105.1 121.4 + 4–21–5 0.02 50 139.1 159.3 140.6 158.8 * 141.8 159.2 + 142.2 159.3 + 8–19–3 100 173.3 198.2 175.6 197.5 * 177.3 198.5 + 179.4 199.3 + 2–25–3 150 173.3 198.2 174.8 197.2 * 177 198.4 + 178.9 199.5 + 2–23–5 0.05 50 275.9 311.8 277.8 311.4 + 278.8 312.3 + 280 313 + 0–28–2 100 357 400.6 364.4 402.6 * 367.1 405 + 369.7 406.7 + 1–28–1 150 357 400.6 364.2 402.7 * 366.8 405.2 + 369.9 407.1 + 0–27–3 0.1 50 489.8 534 490.6 533.7 * 492.6 534 + 493.4 534 + 6–22–2 100 647.8 680.5 643.7 679.3 - 649.5 683.7 + 653 686.4 + 2–24–4 150 648 680.5 642.3 680.9 * 649.6 683.8 + 652.2 687 + 2–22–6 0.2 50 866.9 921.8 867.9 920.6 * 871.5 921.6 + 873.4 921.8 + 1–26–3 100 1120 1182 1120 1181 + 1125 1188 + 1128 1192 + 0–29–1 150 1120 1182 1122 1181 + 1125 1189 + 1129 1193 + 0–30–0

Table 1: Experimental results for cardinality constraints cases. Ranges of final objective values across 30 graphs are shown for each setting. The signed-rank U-tests are used to compare GREEDY’s with GSEMO, GSEMO, GSEMO for each setting, pairing by instances, with 95% confidence level. ‘+’ denotes values being significantly greater than GREEDY’s, ‘-’ denotes less, and ‘*’ denotes no significant difference. Additionally, numbers of losses, wins and ties (L-W-T) GSEMO has over GREEDY are shown, which are determined by separate U-tests on individual instances.

width=center density GREEDY GSEMO GSEMO GSEMO L–W–T min max min max stat min max stat min max stat 50 0.01 2 6.638 13.27 6.625 13.15 - 6.65 13.26 * 6.706 13.27 + 1–7–22 5 6.638 13.27 6.625 13.13 - 6.654 13.25 * 6.706 13.27 + 3–7–20 10 6.638 13.27 6.625 13.15 - 6.65 13.25 * 6.706 13.27 + 1–7–22 0.02 2 13.5 20.27 13.32 20.24 - 13.47 20.28 + 13.5 20.33 + 7–16–7 5 13.5 19.69 13.14 19.53 - 13.4 19.65 + 13.5 19.69 + 7–13–10 10 13.11 20.27 12.94 20.07 - 13.09 20.28 + 13.11 20.33 + 7–14–9 0.05 2 25.8 37.14 26.63 36.82 - 27.07 37.07 + 27.42 37.14 + 5–18–7 5 25.78 36.75 26.49 35.86 - 27.04 36.58 + 27.66 36.88 + 8–18–4 10 25.9 36.83 25.73 35.28 - 27.38 36.17 + 28.09 36.83 + 6–20–4 0.1 2 46.61 57.93 44.87 56.97 - 46.72 57.7 + 47.28 58.05 + 9–18–3 5 45.91 56.69 45.3 55.85 - 46.23 56.4 + 46.81 56.94 + 8–18–4 10 46.59 56.46 44.13 55.73 - 46.41 56.6 + 47.08 57.13 + 6–18–6 0.2 2 78.89 92.5 79.16 91.53 - 80.54 92.18 + 80.82 92.66 + 6–19–5 5 74.37 91.89 74.96 90.83 - 76.44 91.61 + 77.39 91.89 + 6–22–2 10 75.85 92.57 75.62 90.86 - 76.7 92.08 + 77.97 92.51 + 4–21–5 100 0.01 2 27.87 37.79 27.69 37.7 - 28.15 37.99 + 28.67 38.26 + 10–14–6 5 27.87 37.79 27.78 37.65 - 28.1 38 + 28.67 38.26 + 10–14–6 10 27.87 36.81 27.59 36.51 - 27.88 36.91 * 28.31 37.14 + 11–12–7 0.02 2 51.92 66 51.64 65.44 - 52.55 66.17 + 52.79 66.4 + 10–15–5 5 51.62 65.82 51.57 65.48 - 52.41 66.04 * 52.71 66.22 + 8–12–10 10 51.69 63.41 51.38 62.63 - 51.9 63.29 * 52.19 63.75 + 8–13–9 0.05 2 100.1 120.7 99.68 119.3 - 100.9 120.2 + 101.7 120.7 + 8–17–5 5 99.61 119.5 98.19 117.4 - 99.86 119 + 100.6 119.7 + 7–17–6 10 97 116.6 95.82 116 - 98.14 117.6 + 99.37 118.3 + 5–18–7 0.1 2 177.6 197.8 176.8 198.6 * 177.6 199.9 + 178.4 200.7 + 6–19–5 5 173.5 196.6 172.6 197.6 - 174.3 199.1 + 175.8 200.2 + 2–21–7 10 173.3 192.9 171.7 193.9 - 173.6 195.9 * 175.3 198.2 + 12–12–6 0.2 2 294.2 325.1 294.8 324.8 * 297.4 325.9 + 301.6 326.4 + 6–22–2 5 292.7 320.5 293.2 318.5 * 297.3 321.8 + 299.5 323.7 + 3–23–4 10 288.3 322.7 288.2 317.3 - 292.3 321.6 + 296.3 323.8 + 5–22–3 200 0.01 2 103.1 118.4 103.6 118.9 * 104.6 120.2 + 105.1 121.6 + 4–22–4 5 103.1 118.4 103.4 118.5 - 104.2 119.9 + 104.7 121.4 + 3–20–7 10 102.7 117.4 102.6 117.2 - 104 118.1 + 104.7 119.3 + 4–21–5 0.02 2 173.3 198.2 174.8 197.2 * 176.5 198.5 + 178.6 199.5 + 3–22–5 5 172.8 196.7 173.7 195.7 - 176.4 197.6 + 179.1 198.6 + 2–20–8 10 172.5 193.3 173.7 193.9 * 176.1 195.6 + 177.9 196.8 + 1–23–6 0.05 2 356.4 400.6 362.7 401.6 * 366.2 404.8 + 369.1 406.1 + 2–28–0 5 353.7 399.4 359.6 400.5 * 363.5 403 + 365.5 404.7 + 1–28–1 10 352.9 394.3 355.1 396.7 * 360.1 399.4 + 363.2 401.5 + 1–26–3 0.1 2 645.2 680.5 642.8 679 * 647.4 682.9 + 650.7 685.5 + 3–25–2 5 641.7 678.7 637.4 676.2 - 643.3 680.6 + 647.4 685.2 + 4–20–6 10 637.2 669.8 632.3 667.4 - 641.5 672.7 + 645.1 677.3 + 2–24–4 0.2 2 1119 1182 1118 1183 + 1123 1187 + 1128 1193 + 0–30–0 5 1117 1178 1116 1173 * 1121 1179 + 1125 1184 + 0–29–1 10 1105 1170 1111 1175 * 1117 1181 + 1122 1188 + 2–28–0

Table 2: Experimental results for partition matroid constraints cases. Ranges of final objective values across 30 graphs are shown for each setting. The signed-rank U-tests are used to compare GREEDY’s with GSEMO, GSEMO, GSEMO for each setting, pairing by instances, with 95% confidence level. ‘+’ denotes values being significantly greater than GREEDY’s, ‘-’ denotes less, and ‘*’ denotes no significant difference. Additionally, numbers of losses, wins and ties (L-W-T) GSEMO has over GREEDY are shown, which are determined by separate U-tests on individual instances.

The experimental results for cardinality constraint cases are shown in Table 1. Signed-rank U-tests [9] are applied to the outputs, with pairing based on instances. Furthermore, we count the numbers of instances where GSEMO outperforms, ties with, and is outperformed by GREEDY via separate U-tests on individual instances.

Overall, GSEMO on average outperforms GREEDY with statistical significance in most cases. The significance seems to increase, with some noises, with increasing , density, and . This indicates that GSEMO more reliably produces better solutions than GREEDY as the graph’s size and density increase. Moreover, in few cases with large , GSEMO is higher than GREEDY’s with statistical significance.

Additionally, it is indicated that GSEMO tend to be closer to GSEMO than GSEMO

. This suggests skewed distribution of outputs in each instance toward high values. The implication is that GSEMO is more likely to produce outputs greater than average, than otherwise. It might be an indication that these results are close to the global optima for these instances.

Per instance analyses show high number of ties between GSEMO and GREEDY for small , density, and to a lesser extent . As these increase, the number of GSEMO’s wins increases and ends up dominating at . This trend coincides with earlier observations, and suggests the difficulty of making improvements in sparse graphs faced by GSEMO where GREEDY heuristic seems more suited. On the other hand, large graph sizes seem to favour GSEMO over GREEDY despite high sparsity, likely due to more local optima present in larger search spaces.

4.3 Cut maximization under partition matroid constraints

The experimental results for partition matroid constraints cases are shown in Table 2. Notations and statistical test procedure are the same as in Table 1.

Overall, the main trend in cardinality constraint cases is present: GSEMO on average outperforms GREEDY, with increased reliability at larger and density. This can be observed in both the average performances and the frequency at which GSEMO beats GREEDY. It seems the effect of this phenomenon is less influenced by variations in than it is by variations in in cardinality constraint cases. Note that the analysis in Theorem 1 only considers bottom-up improvements up to . Experimental results likely suggest GSEMO can make similar improvements beyond that point up to .

Additionally, the outputs of both algorithms are generally decreased with increased , due to restricted feasible solution spaces. There are few exceptions which could be attributed to noises from random partitioning since they occur in both GREEDY’s and GSEMO’s simultaneously. Furthermore, the variation in seems to slightly affect the gaps between GSEMO’s and GREEDY’s, which are somewhat smaller at higher . It seems to support the notion that GREEDY performs well in small search spaces while GSEMO excels in large search spaces. This coincides with the observations in the cardinality constraint cases, and explains the main trend.

5 Conclusion

In this work, we consider the problem of maximizing a set function under partition matroid constraints, and analyze GSEMO’s approximation performance on such problems. Theoretical performance guarantees are derived for GSEMO in cases of submodular objective functions, and monotone objective functions. We show that GSEMO guarantees good approximation quality within polynomial expected run time in both cases. Additionally, experiments with Max Cut instances generated from varying settings have been conducted to gain insight on its empirical performance, based on comparison against simple GREEDY’s. The results show that GSEMO generally outperforms GREEDY within quadratic run time, particularly when the feasible solution space is large.

Acknowledgements

The experiments were run using the HPC service provided by the University of Adelaide.

References

  • [1] Auger, A, Doerr, B. (Eds.) (2011) Theory of randomized search heuristics: foundations and recent developments. World Scientific Publishing Co., Inc.. Cited by: §1.
  • [2] M. Balcan, A. Blum, and Y. Mansour (2008) Item pricing for revenue maximization. In Proceedings of the 9th ACM Conference on Electronic Commerce, EC ’08, New York, NY, USA, pp. 50–59. External Links: ISBN 978-1-60558-169-9, Document Cited by: §1.2.
  • [3] K. Bhawalkar and T. Roughgarden (2011-10) Welfare guarantees for combinatorial auctions with item bidding. In Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’11, Philadelphia, PA, USA, pp. 700–709. External Links: Document Cited by: §1.2.
  • [4] A. A. Bian, J. M. Buhmann, A. Krause, and S. Tschiatschek (2017) Guarantees for greedy maximization of non-submodular functions with applications. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, pp. 498––507. Cited by: §1.1.
  • [5] N. Buchbinder, M. Feldman, and M. Garg (2019) Deterministic -approximation for submodular maximization over a matroid. In Proceedings of the 30th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’19, Philadelphia, PA, USA, pp. 241–254. Cited by: §1.1.
  • [6] G. Călinescu, C. Chekuri, M. Pál, and J. Vondrák (2011-12) Maximizing a monotone submodular function subject to a matroid constraint. SIAM Journal on Computing 40 (6), pp. 1740–1766. External Links: ISSN 0097-5397, Document Cited by: §1.1.
  • [7] C. Chekuri and A. Kumar (2004) Maximum coverage problem with group budget constraints and applications. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, K. Jansen, S. Khanna, J. D. P. Rolim, and D. Ron (Eds.), Berlin, Heidelberg, pp. 72–83. Cited by: §1.1.
  • [8] M. Conforti and G. Cornuéjols (1984) Submodular set functions, matroids and the greedy algorithm: tight worst-case bounds and some generalizations of the rado-edmonds theorem. Discrete Applied Mathematics 7 (3), pp. 251–274. External Links: ISSN 0166-218X, Document Cited by: §1.1.
  • [9] G. W. Corder and D. I. Foreman (2009) Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach. Wiley. Note: Hardcover External Links: ISBN 047045461X Cited by: §4.2.
  • [10] G. Cornuejols, M. L. Fisher, and G. L. Nemhauser (1977-04) Location of bank accounts to optimize float: an analytic study of exact and approximate algorithms. Management Science 23 (8), pp. 789–810. External Links: ISSN 00251909, 15265501, Document Cited by: §1.1.
  • [11] A. Das and D. Kempe (2011) Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, Madison, WI, USA, pp. 1057–1064. External Links: ISBN 9781450306195 Cited by: §2.1.
  • [12] Doerr, B., Neumann, F. (Eds.) (2020) Theory of evolutionary computation – recent developments in discrete optimization. Natural Computing Series, Springer. External Links: Document, ISBN 978-3-030-29413-7 Cited by: §1.
  • [13] U. Feige, V. S. Mirrokni, and J. Vondrák (2011-07) Maximizing non-monotone submodular functions. SIAM Journal on Computing 40 (4), pp. 1133–1153. External Links: ISSN 0097-5397, Document Cited by: §1.2.
  • [14] L. Fleischer, M. X. Goemans, V. S. Mirrokni, and M. Sviridenko (2006-01) Tight approximation algorithms for maximum general assignment problems. In Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithm, SODA ’06, Philadelphia, PA, USA, pp. 611–620. External Links: ISBN 0-89871-605-5, Document Cited by: §1.1.
  • [15] T. Friedrich, A. Göbel, F. Neumann, F. Quinzan, and R. Rothenberger (2019-07) Greedy maximization of functions with bounded curvature under partition matroid constraints.

    Proceedings of the AAAI Conference on Artificial Intelligence

    33, pp. 2272–2279.
    External Links: Document Cited by: §1.1, §1.1, §1.2, §1.2, §2.2, §3.
  • [16] T. Friedrich, J. He, N. Hebbinghaus, F. Neumann, and C. Witt (2010-12) Approximating covering problems by randomized search heuristics using multi-objective models*. Evolutionary Computation 18 (4), pp. 617––633. External Links: ISSN 1063-6560, Document Cited by: §2.2.
  • [17] T. Friedrich and F. Neumann (2015-12) Maximizing submodular functions under matroid constraints by evolutionary algorithms. Evolutionary Computation 23 (4), pp. 543–558. External Links: Document, ISSN 1063-6560 Cited by: §1.1, §2.2, §2.2, §2.2, §3.
  • [18] M. X. Goemans and D. P. Williamson (1995-11) Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM 42 (6), pp. 1115––1145. External Links: ISSN 0004-5411, Document Cited by: §1.1.
  • [19] S. Iwata, L. Fleischer, and S. Fujishige (2001-07) A combinatorial strongly polynomial algorithm for minimizing submodular functions. Journal of the ACM 48 (4), pp. 761–777. External Links: ISSN 0004-5411, Document Cited by: §1.1.
  • [20] T. Jansen (2013) Analyzing evolutionary algorithms - the computer science perspective. Natural Computing Series, Springer. External Links: ISBN 978-3-642-17338-7 Cited by: §1.
  • [21] S. Jegelka and J. Bilmes (2011-06) Submodularity beyond submodular energies: coupling edges in graph cuts. In CVPR 2011, CVPR ’11, Vol. , pp. 1897–1904. External Links: ISSN 1063-6919, Document Cited by: §1.1.
  • [22] D. Kempe, J. Kleinberg, and É. Tardos (2003) Maximizing the spread of influence through a social network. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’03, New York, NY, USA, pp. 137–146. External Links: ISBN 1-58113-737-0, Document Cited by: §1.1.
  • [23] A. Krause and C. Guestrin (2011-07) Submodularity and its applications in optimized information gathering. ACM Transactions on Intelligent Systems and Technology 2 (4), pp. 32:1–32:20. External Links: ISSN 2157-6904, Document Cited by: §1.1.
  • [24] A. Krause, A. Singh, and C. Guestrin (2008-02) Near-optimal sensor placements in gaussian processes: theory, efficient algorithms and empirical studies. Journal of Machine Learning Research 9, pp. 235–284. External Links: ISSN 1532-4435, Document Cited by: §1.1, §2.1.
  • [25] M. Laumanns, L. Thiele, and E. Zitzler (2004-04) Running time analysis of multiobjective evolutionary algorithms on pseudo-boolean functions. IEEE Transactions on Evolutionary Computation 8 (2), pp. 170–182. External Links: Document, ISSN 1941-0026 Cited by: §2.2.
  • [26] J. Lee, V. S. Mirrokni, V. Nagarajan, and M. Sviridenko (2009-02) Non-monotone submodular maximization under matroid and knapsack constraints. In

    Proceedings of the 41st Annual ACM Symposium on Theory of Computing

    ,
    STOC ’09, New York, NY, USA, pp. 323–332. External Links: ISBN 978-1-60558-506-2, Document Cited by: §1.1.
  • [27] J. Lengler (2017) Drift analysis. CoRR abs/1712.00964. External Links: Link, 1712.00964 Cited by: §3, §3.
  • [28] H. Lin and J. Bilmes (2010)

    Multi-document summarization via budgeted maximization of submodular functions

    .
    In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT ’10, USA, pp. 912––920. External Links: ISBN 1932432655 Cited by: §1.1.
  • [29] H. Lin and J. Bilmes (2011-06) A class of submodular functions for document summarization. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, HLT ’11, Portland, Oregon, USA, pp. 510–520. Cited by: §1.1.
  • [30] Y. Liu, K. Wei, K. Kirchhoff, Y. Song, and J. Bilmes (2013-05)

    Submodular feature selection for high-dimensional acoustic score spaces

    .
    In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vol. , pp. 7184–7188. External Links: Document, ISSN 1520-6149 Cited by: §1.1.
  • [31] L. Lovász (1983) Submodular functions and convexity. In Mathematical Programming The State of the Art: Bonn 1982, A. Bachem, B. Korte, and M. Grötschel (Eds.), pp. 235–257. External Links: ISBN 978-3-642-68874-4, Document Cited by: §1.1.
  • [32] G. L. Nemhauser and L. A. Wolsey (1978-08) Best algorithms for approximating the maximum of a submodular set function. Mathematics of Operations Research 3 (3), pp. 177–188. External Links: ISSN 0364-765X, Document Cited by: §2.1, §2.2.
  • [33] F. Neumann and C. Witt (2010) Bioinspired computation in combinatorial optimization. Natural Computing Series, Springer. External Links: Link, Document, ISBN 978-3-642-16543-6 Cited by: §1.
  • [34] C. Qian, J. Shi, Y. Yu, K. Tang, and Z. Zhou (2016) Parallel pareto optimization for subset selection. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, pp. 1939–1945. External Links: ISBN 9781577357704 Cited by: §3.
  • [35] C. Qian, J. Shi, Y. Yu, and K. Tang (2017-08) On subset selection with general cost constraints. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI-17, pp. 2613–2619. External Links: Document Cited by: §2.2, §2.2, §2.2, §3.
  • [36] C. Qian, Y. Yu, K. Tang, X. Yao, and Z. Zhou (2019) Maximizing submodular or monotone approximately submodular functions by multi-objective evolutionary algorithms. Artificial Intelligence 275, pp. 279–294. External Links: ISSN 0004-3702, Document Cited by: §1.1, §1.2, §3, §3, Lemma 1.
  • [37] M. Queyranne (1995) A combinatorial algorithm for minimizing symmetric submodular functions. In Proceedings of the 6th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’95, USA, pp. 98–101. External Links: ISBN 0898713498 Cited by: §1.2.
  • [38] P. Stobbe and A. Krause (2010-10) Efficient minimization of decomposable submodular functions. In Proceedings of the 23rd International Conference on Neural Information Processing Systems - Volume 2, NIPS ’10, USA, pp. 2208–2216. Cited by: §1.1.
  • [39] V. Syrgkanis and É. Tardos (2013) Composable and efficient mechanisms. In Proceedings of the 45th Annual ACM Symposium on Theory of Computing, STOC ’13, New York, NY, USA, pp. 211–220. External Links: ISBN 978-1-4503-2029-0, Document Cited by: §1.2.
  • [40] J. Vondrák (2010-01) Submodularity and curvature: the optimal algorithm. RIMS Kôkyûroku Bessatsu B23, pp. 253–266. Cited by: §1.1.
  • [41] K. Wei, R. Iyer, and J. Bilmes (2015)

    Submodularity in data subset selection and active learning

    .
    In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML ’15, pp. 1954–1963. Cited by: §1.1.
  • [42] Z. Zhou, Y. Yu, and C. Qian (2019) Evolutionary learning: advances in theories and algorithms. Springer. External Links: Link, Document, ISBN 978-981-13-5955-2 Cited by: §1.