 # Diversity in Combinatorial Optimization

When modeling an application of practical relevance as an instance of a combinatorial problem X, we are often interested not merely in finding one optimal solution for that instance, but in finding a sufficiently diverse collection of good solutions. In this work we introduce an intuitive notion of diversity of a collection of solutions which suits a large variety of combinatorial problems of practical interest. We then present an algorithmic framework which---automatically---converts a tree-decomposition-based dynamic programming algorithm for a given combinatorial problem X into a dynamic programming algorithm for the diverse version of X. Surprisingly, our algorithm has a polynomial dependence on the diversity parameter. Going further, we devise a framework to translate kernels of a certain type for a given combinatorial problem X into kernels of a slightly larger size for its diverse version.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

In a typical combinatorial optimization problem we are given a large space of potential solutions and an objective function. The task is to find a solution which maximizes or minimizes the objective function. In many situations of practical relevance, however, it does not really help to get just one optimal solution; it would be much better to have a small, but sufficiently diverse collection of sufficiently good solutions. Given such a small list of good solutions we can select one which is best for our purpose, perhaps by taking into account external factors—such as aesthetic, political, or environmental—which are difficult or impossible to formalize. An early, illustrative example is the problem of generating floor plans for evaluation by an architect .

In this work, we deal with vertex-problems on graphs. Formally, a vertex-problem is a set of pairs of the form , where is an undirected graph and is a subset of vertices of . For an illustration, recall that a vertex cover in a graph is any subset of vertices of such that for each edge of at least one endpoint of belongs to . For each , the vertex-problem VC is defined to be the set of all pairs of the form where is a graph and is a vertex cover in of size at most . A large variety of combinatorial optimization problems can similarly be formalized as vertex-problems on graphs .

Vertex cover can be used to model several resolution conflict scenarios, and for this reason has been applied in several fields of expertise, such a s as computational biology , biochemistry , computer network security , and classification methods . One way to abstractly model a conflict scenario is to consider a set of elements and between each pair of elements, there is potentially a conflict. The goal is to make the set conflict-free, and one way to resolve a conflict is to remove one of its elements from the set. Naturally, we would want to remove as few elements as possible to make the set conflict-free. Now, taking a graph whose vertex set is the set of elements and whose edges represent the conflicts, the smallest set whose removal results in a conflict-free subset is precisely a vertex cover of that graph.

#### A Formal Notion of Diversity.

We choose a very natural and general measure as our notion of diversity among solutions. Given two subsets and of a set the Hamming distance between and is the number

 HamDist(S,S′)=|S∖S′|+|S′∖S|.

We define the diversity of a list of subsets of to be

 Div(S1,…,Sr)=∑1≤i

We can now define the diverse version of vertex-problems:

###### Definition 1 (Diverse Problem).

Let be vertex-problems, and let . We let

 Divd(P1,…,Pr)={(G,X1,…,Xr)∣(G,Xi)∈Pi,Div(X1,…,Xr)≥d}.

Intuitively, given vertex-problems and a graph , we want to find subsets of vertices of such that for each , is a solution for problem on input , and such that the list has diversity at least . If all vertex-problems are the same problem , then we write as a shortcut to .

#### Diversity and Dynamic Programming.

The treewidth of a graph is a structural parameter that quantifies how close the graph is to being a forest (i.e., a graph without cycles). The popularity of this parameter stems from the fact that many problems that are NP-complete on general graphs can be solved in polynomial time on graphs of constant treewidth. In particular, a celebrated theorem due to Courcelle 

states that any problem expressible in the monadic second-order logic of graphs can be solved in polynomial time on graphs of constant treewidth. Besides this metatheorem, the notion of treewidth has found applications in several branches of Artificial Intelligence such as Answer Set Programs

, checking the consistency of certain relational algebras in Qualitative Spacial Reasoning 

, compiling Bayesian networks

, determining the winners of multiwinner voting systems , analyzing the dynamics of stochastic social networks , and solving constraint satisfaction problems . The running times of many of these algorithms are of the form where is the number of vertices and the treewidth of the input graph, and is a computable function of alone. In other words, these algorithms are fixed-parameter tractable (or FPT for the parameter treewidth. Additionally, these algorithms are dynamic programming algorithms which operate on a tree-decomposition in a bottom-up fashion by computing data from the leaves to the root.

Our main result (Theorem 10) is a framework to efficiently—and automatically—transform treewidth-based dynamic programming algorithms for vertex-problems into algorithms for the diverse versions of these problems. More precisely, we show that if are vertex-problems where for each , can be solved in time then can be solved in time . In particular, if a vertex-problem can be solved in time , then its diverse version can be solved in time . The surprising aspect of this result is that the running time depends only polynomially on (which is at most ), while a naïve dynamic programming algorithm would have an extra multiplicative factor of in the running time.

#### Diversity in Kernelization

A kernelization is a polynomial-time algorithm which takes as input an instance of a parameterized problem and outputs an equivalent instance whose size is bounded by a function of the parameter alone [12, 14, 18]. If that function is a polynomial, we speak of a polynomial kernel. In parameterized complexity, once a problem has been shown to be FPT with respect to a parameter, the next step is to try to determine whether it has a polynomial kernel111It is known that a problem is FPT if and only if it has a kernelization, and that some problems do not admit polynomial kernels unless (See, e.g., ).. The study of the kernelization complexity of parameterized problems has long arrived in Artificial Intelligence research; this includes work on upper and lower bounds on kernel sizes for constraint satisfaction and related problems [7, 21, 22, 30], planning [2, 26] and reasoning  problems, and network security [1, 28]. Hebrard has recently advocated the use of kernelization techniques to attack problems in constraint satisfaction , and some empirical success has already been reported .

In [8, 7] it was shown that many constraint satisfaction problems admit the so-called loss-less polynomial kernels. In Section 5 we show that if a subset minimization problem admits a loss-less polynomial kernel then its diverse version admits a kernel of size bounded by a polynomial function of the original parameter and the number of requested solutions. Using this we show that the diverse variants of several well-studied combinatorial problems admit such polynomial kernels.

## 2 Preliminaries

For positive integers we use to denote the set . We use and , respectively, to denote the vertex and edge sets of a graph . For a tree rooted at we use to denote the subtree of rooted at a vertex . A rooted tree decomposition of a graph is a tuple , where is a tree rooted at and is a collection of subsets of such that:

• ,

• for every edge , there is a such that , and

• for each such that lies on the unique path between and in , .

We say that the vertices of are the nodes of and that the sets in are the bags of . Given a node , we denote by the subgraph of induced by the set of vertices

 ⋃s∈V(subtree(T,t))Xs.

The width of a tree decomposition is defined as . The treewidth of a graph , denoted by , is the smallest integer such that there exists a rooted tree decomposition of of width at most . The rooted path decomposition of a graph is a rooted tree decomposition such that is a path and is a vertex of degree . The pathwidth of a graph , denoted by , is the smallest integer such that there exists a rooted path decomposition of of width at most . Note that in a rooted path decomposition, every node as at most one child.

For convenience we will always assume that the bag associated to the root of a rooted tree decomposition is empty. For a node we use to denote the number of children of in the tree . For nodes and of where is the parent of we use to denote the set of vertices of which are forgotten at . By convention, for the root of , we let . For each we denote by the set where are the children of .

Given a rooted tree decomposition of a graph one can obtain, in linear time, a tree decomposition of of the same width as such that for each , and  . From now on we assume that every rooted tree decomposition is of this kind.

#### Extra problem definitions.

In the -Hitting Set problem, we are given a hypergraph , each of whose hyperedges contains at most elements, and an integer , and the goal is to find a set of vertices of of size at most such that each hyperedge contains at least one element from . In the Point Line Cover problem, we are given a set of points in the plane and an integer and we want to find a set of at most lines such that each point lies on at least one of the lines. A directed graph is called a tournament, if for each pair of vertices , either the edge directed from to or the edge directed from to is contained in the set of arcs of . In the Feedback Arc Set in Tournaments problem we are given a tournament and an integer and the goal is to find a set of at most arcs such that after removing this set, the resulting directed graph does not contain any directed cycles.

## 3 A First Example: Diverse Vertex Cover

The main result of this paper is a general framework to automatically translate tree-decomposition-based dynamic programming algorithms for vertex-problems into algorithms for the diverse versions of these problems. We develop this framework in Section 4. In this section we illustrate the main techniques used in this conversion process by showing how to translate a tree-decomposition-based dynamic programming algorithm for the Vertex Cover problem into an algorithm for its diverse version Diverse Vertex Cover. Given a graph and three integers , , and , the Diverse Vertex Cover problem asks whether one can find vertex covers in , each of size at most , such that their diversity is at least . Our algorithm for this problem will run in time.

### 3.1 Incremental Computation of Diversity

Recall that we defined the diversity of a list of subsets of a set to be

 Div(S1,…,Sr)=∑1≤i

We will now describe a way to compute the diversity in an incremental fashion, by incorporating the influence of each element of in turn. For each element and each pair of subsets of , we define to be if , and to be otherwise. Intuitively, is if and only if the element contributes to the Hamming distance between and . Given this definition we can rewrite as

 HamDist(S,S′)=∑v∈Vγ(S,S′,v),

and the diversity of a list of subsets of as

 Div(S1,…,Sr)=∑1≤i

Now, if we let

 I(S1,…,Sr,v)=|{ℓ:v∈Sℓ}|⋅|{ℓ:v∉Sℓ}|,

then we have that

 Div(S1,…,Sr)=∑v∈VI(S1,…,Sr,v). (1)

### 3.2 From Vertex Cover to Diverse Vertex Cover

We now solve Diverse Vertex Cover using dynamic programming over a tree decomposition of the input graph. An excellent exposition of tree-width-based dynamic programming algorithms can be found in [12, Chapter 7].

Let be an instance of Diverse Vertex Cover and let be a rooted tree decomposition of . For each node , we define the set

 It={((S1,s1),…,(Sr,sr),ℓ)∣ℓ∈[0,d],∀i∈[1,r],Si⊆Xt,si∈[0,k]}.

Note that for each , . Now, our dynamic programming algorithm for Diverse Vertex Cover consists in constructing for each a subset as follows. Let be a node in with children . We recall that, by convention, this set of children is of size , , or . We let be the set of all tuples satisfying the following additional properties:

1. For each , .

2. For each there exists a tuple in such that

1. for each and each ,

2. For each , ,

3. and where

###### Lemma 2.

is a Yes-instance of Diverse Vertex Cover if and only if there is a tuple in such that .

###### Proof.

Using induction, one can see that for each , is the set of every element of such that, with , there exists , that satisfies:

• for each , is a vertex cover of ,

• for each , ,

• for each , , and

• .

As the root of the tree decomposition is such that , we obtain that the element in are the elements of such that there exists , that satisfy,

• for each , is a vertex cover of ,

• for each , , and

• .

As such, a tuple of subsets of is a solution of Diverse Vertex Cover if and only if , the lemma follows. ∎

###### Theorem 3.

Given a graph , integers , and a rooted tree decomposition of of width , one can determine whether is a Yes-instance of Diverse Vertex Cover in time

 O(2r⋅(2w+1⋅(k+1))a⋅r⋅da⋅w⋅r⋅n),

where and .

###### Proof.

Let us analyze the time needed to compute . We have that, for each , . Note that given be elements of , there are at most way to create an element of by selecting, or not the (potential) new element of for each set , . The remaining is indeed fixed by . Thus, can be computed in time , where the factor appears when verifying that the element we construct satisfy . As we need to compute for each and that and we can assume that for each , the theorem follows. ∎

###### Remark 4.

Given a graph and a vertex cover of of size , one can find a rooted path decomposition of of width , in linear time.

This can be done by considering the bags for each in any fixed order. Thus, from Theorem 3, we get the following corollary, which establishes an upper bound for the running time of our dynamic programming algorithm for Diverse Vertex Cover solely in terms of the size of the vertex cover, the number of requested solutions, and the diversity .

###### Corollary 5.

Diverse Vertex Cover can be solved on an input in time

 O((2k+2⋅(k+1))r⋅d⋅k⋅r⋅|V(G)|).

## 4 Computing Diverse Solutions by Dynamic Programming over Tree Decompositions

In this section we introduce a general framework to convert tree-width-based dynamic programming algorithms for vertex-problems into dynamic programming algorithms for the diverse versions of these problems. We start by defining a very general notion of tree-width-based dynamic programming. Below, we let be the set of simple, undirected graphs whose vertex set is a finite subset of . We say that a subset is a graph problem. A tree-width-based dynamic programming algorithm for a graph problem can be understood as a procedure that takes a graph and a rooted tree decomposition of , and constructs a certain amount of data for each node of . This data at node is constructed by induction on the height of , and in general, this data is used to encode the existence of a partial solution on the graph induced by bags in the subtree of rooted at . Such an algorithm accepts the input graph if the data associated with the root node contains a string belonging to a set of accepting strings. This approach can be formalized using the notion of a dynamic core, which we now define.

###### Definition 6 (Dynamic Core).

A dynamic core is an algorithm that takes a graph and a rooted tree decomposition of as input, and produces the following data.

• A finite set .

• A finite set for each .

We let be the overall time necessary to construct the data associated with all nodes of . The size of on a pair is defined as

 Size(C,G,D)=max{|ProcessC,G,D(t)|∣t∈V(D)}.
###### Definition 7.

Let be a dynamic core, be a graph in , and be a rooted tree decomposition of . A -witness is a function such that the following conditions are satisfied for each .

1. For each , with children , .

2. .

###### Definition 8.

We say that a dynamic core solves a problem if for each graph , and each rooted tree decomposition of , if and only if a -witness exists.

###### Theorem 9.

Let be a graph problem and be a dynamic core that solves . Given a graph and a rooted tree decomposition of , one can decide whether in time .

###### Proof.

Given , , and , we construct and , . By definition, this can be done in time .

Given and , a -witness is a function

 β:V(subtree(T,t))→{0,1}∗

such that for each , with children , and . Note that there exists a -witness if and only if there exists a -witness for some .

For each , we define to be the set of every such that there exists a -witness. Let and assume that we are able to construct for every where are the children of . We can then construct as follows. For each , we add to if for each , . It is easy to see that for each such , there exists a -witness that is an extension of the -witness, . Moreover if there exists a -witness for some , then, for each , the restriction of to is a -witness for some , and so, by induction hypothesis, . This implies that our construction has correctly added to . Thus is correctly constructed.

From Definition 8 we have that if and only if . Notice that the time needed to construct is . The theorem follows. ∎

### 4.1 Dynamic Cores for Vertex-Problems

Let be a dynamic core. A -vertex-membership function is a function such that for each graph , each rooted tree decomposition of and each -witness , it holds that for each edge and each vertex . Intuitively, if is a graph and is a rooted tree decomposition of , then a -vertex-membership together with a -witness, provide an encoding of a subset of vertices of the graph. More precisely, we let

 Sρ(G,D,α)={v∣∃t∈V(TD),ρ(v,α(t))=1}

be this encoded vertex set. Given a -vertex-membership function , we let be the function that sets for each .

Let be a vertex-problem, be a dynamic core, and be a -vertex-membership function. We say that solves if for each graph , each subset , and each rooted tree decomposition , if and only if there exists a -witness such that .

###### Theorem 10.

Let be vertex-problems, let be a dynamic core for , and let be an integer. , on graph with rooted tree decomposition , can be solved in time , where .

Let and . We extend the definition of diverse influence to such that

 I(w1,…,wr,v)=I(^ρ1(w1),…,^ρr(wr),v).

The proof of Theorem 10 uses the following lemma.

###### Lemma 11.

Let be a graph and be a rooted tree decomposition of . belongs to if and only if there exist such that the following conditions are satisfied.

1. For each , is a -witness and .

2. .

###### Proof.

First assume that belongs to . By Definition 1, for each , we have that , and so, there exists a -witness such that . Thus Condition 1 is satisfied. Moreover, we have that for each and each , . Together with the fact that each vertex is in exactly one set , , and imply Condition 2.

Assume now that there exist that satisfy Conditions 1 and 2. Condition 1 implies that for each , . Moreover, as for each , there is exactly one node such that , by definition of a rooted tree decomposition, Condition 2 implies that . Thus, belongs to . ∎

We can now provide the proof of Theorem 10.

###### Proof of Theorem 10.

For each , we start by constructing the data corresponding to the dynamic core . The overall construction takes time .

Now, we define a dynamic core for the problem . Let and be a rooted tree decomposition of . produces the following data.

• .

• for each .

Let be a -witness of , let be the projection of to its -th coordinate, and let be the projection of to its last coordinate. Then we have that is a -witness for if and only if is a -witness for , and for being the root of ,

 β(q)=min{d,∑t∈V(D)∑v∈forg(t)I(α1(t),…,αr(t),v)}≥d.

By Lemma 11, we have that this happens if and only if

 (G,Sρ1(G,D,α1),…,Sρr(G,D,αr))

belongs to .

Let now analyze the running time of this procedure. When constructing for some , we need to combine every combinaison of element of , and of values of , . This can be done in time . Thus constructing the data associated to , , and takes

Moreover, as for every , , then by Theorem 9, can be solved in time where . The theorem follows. ∎

We describe now how to use Theorem 10 for Diverse Vertex Cover. First we need to describe what is a dynamic core that solves Vertex Cover of size at most . Given a graph and a rooted tree decomposition , this dynamic core produces:

• and

• , for each , where are the children of .

Provided the width of the decomposition is at most , this can be done in time for each , where the factor appears as we need the conditions and to be verified. It is easy to verify that is a dynamic core for the Vertex Cover problem. As describe in Remark 4, we know that we can construct a rooted path decomposition of of width . We are now considering this rooted path decomposition. Thus, for each , . By Theorem 10, we obtain the following corollary, improving Corollary 5.

###### Corollary 12.

Diverse Vertex Cover can be solved on an input in time

 O(d⋅|V(G)|⋅(2k+2⋅(k+1))r+|V(G)|⋅2k+1⋅(k+1)⋅k).

Note that we obtain a slightly better running time than for Corollary 5. This is due to the fact that verifying the properties and is done when constructing and not when constructing . Note also that, formally, we need to construct times