Given a graph , the vertex expansion of a non-empty subset , denoted by , is defined as333 Other definitions of vertex expansion have been studied in the literature, see sec:relatedwork.
where , the neighborhood of , is defined as . The vertex expansion of the graph , denoted by , is defined as . Computing the vertex expansion of a graph is NP-hard. The complexity of computing various graph expansion parameters are central open problems in theoretical computer science, and inspite of many decades of intensive research, they are yet to be fully understood [Alo86, AM85, LR99, ARV09, FHL08, RS10].
Feige et. al. [FHL08] gave a -approximation algorithm for computing the vertex expansion of a graph. Louis et. al. [LRV13] gave an algorithm that computes a set having vertex expansion at most in graphs having vertex degrees at most . We give a brief description of other related works in sec:relatedwork. In this work, we study a natural semi-random family of graphs, and give polynomial time exact and approximation algorithms for computing the balanced vertex expansion (a notion that is closely related to the vertex expansion of a graph, we define it formally in sec:model) w.h.p.
In many problems, there is a huge gap betwen theory and practice; the best known algorithms provide a somewhat underwhelming performance guarantee, however simple heuristics perform remarkably well in practice. Examples of this include the simplex algorithm for linear programming[KM72], SAT [BP99], sparsest cut [KK95, KK98], among others. In many cases, the underwhelming provable approximation guarantee of an algorithm is a property (hardness of approximation) of the problem itself; even in many such cases, simple heuristics work remarkably well in practice. A possible explanation for this phenomenon could be that for many problems, the instances arising in practice tend to have some inherent structure that makes them “easier” than the worst case instances. Many attempts have been made to understand the structural properties of these instances, and to use them in designing algorithms specifically for such instances, which could perform much better than algorithms for general instances. A fruitful direction of study has been that of modelling real world instances as a family of random and semi-random instances satisfying certain properties. Our work can be viewed as the study of the computation of vertex expansion along this direction.
Often graphs with sparse cuts are used to model communities. For example, the vertices of a graph can be used to represent the members of the communities, and two vertices would have an edge between them if the members corresponding to them are related in some way. In such a graph, the sparse cuts indicate the presence of a small number of relations across the members corresponding to the cut, which are likely to be some form of communities within the members. The stochastic block models have been used to model such communities. Our model can also be viewed as model for communities where only a few members from each community have a relationship with members from another community.
We define our model in sec:model, and state our results in sec:results. We give our SDP relaxation in sec:sdp. We give an overview of our proofs in sec:proofoverview, and present the proofs of our theorems in sec:proof and sec:const-factor-approx.
1.1 Vertex Expansion Block Models.
For a graph , its balanced vertex expansion is defined as
Another common notion of vertex expansion that has been studied in the literature is , and as before, . [LRV13] showed that the computation and is equivalent upto constant factors. In this work, we develop a semi-random model for investigating the balanced vertex expansion of graphs.
We study instances that are constructed as follows. We start with a set of vertices, and
we arbitrarily partition them into two sets , of vertices each. Next, we choose
a small subset of size (resp. ) to form the vertex boundary of these sets.
On (resp. ), we add an arbitrary graph whose spectral gap444 The spectral gap of a graph is defined as the second smallest eigenvalue of its normalized Laplacian matrix,
see sec:notation for definition
The spectral gap of a graph is defined as the second smallest eigenvalue of its normalized Laplacian matrix, see sec:notation for definitionis at least (a parameter in this model), and whose vertices have roughly the same degree. We add an arbitrary low degree bipartite graph between and . Between each pair of vertices in
, we add edges independently at random with probability; this is the only part of the construction that is random. Next, we allow a monotone adversary to alter the graph : the monotone adversary can arbitrarily add edges that do not change the sparsity of the vertex cut , i.e., add edges between any pair of vertices in (resp. ), and between any pair in .
In our model, we allow the sets and to be generated using different sets of parameters, i.e., we use for and for . We formally define the vertex expansion block model below (see also fig:model); we refer to it as the VBM model.
Definition 1.1 (The Vbm model).
An instance of VBM is generated as follows.
Let be a set of vertices. Partition into two sets and of vertices each. Partition into two sets and of sizes and respectively. Similarly, partition into two sets and of sizes and respectively.
Between each pair in (resp. ), add an edge independently with probability (resp. ).
Between pairs of vertices in (resp. ), add edges to form an arbitrary roughly regular (formally, ratio of the maximum vertex degree and the minimum vertex degree is at most ) of spectral gap4 at least .
Between pairs in , add edges to form an arbitrary bipartite graph of vertex degrees in the range (this bipartite graph need not be connected); if , then add no edges in this step. We will use to denote this bipartite graph.
(Monotone Adversary) Arbitrarily add edges between any pair of vertices in (resp. ). Arbitrarily add edges between any pair in .
Output the resulting graph .
We note that the direct analogue for vertex expansion of Stochastic Block Models (see related work in Section 1.3) in the regimes allowing for exact recovery is included in this setting: there, the graphs within and are completely random, and so are the connections between and (before the monotone adversary acts). Our model allows for a lot more adversarial action, while restricting the randomness to only a small portion of the graph.
In addition to being a family of instances that will help us to better understand the complexity of the computation of vertex expansion, the vertex expansion block model can also be used in the study of community detection. In the case of two communities, the vertices in and can model the members of the communities. Each community can have a few representatives who interact with the representatives from other communities; these representatives can be modelled using and , and their interactions can be modelled by the arbitrary graphs within and , and the low degree bipartite graph and the action of the monotone adversary between and . Even though the connections within a community may be arbitrary, usually the members within the community are well connected with each other; this can be modelled by the choosing an appropriate values of plus the action of the monotone adversary. We can model the connections between community members and their representatives by a sparse random bipartite graph; our model allows the flexibility of choosing and , and also the action of the monotone adversary.
1.2 Our Results
Our main result is a polynomial time algorithm for exactly recovering and from a graph sampled from VBM for certain ranges of parameters.
There exist universal constants satisfying the following: there exists a polynomial time algorithm which takes a graph generated from VBM, where , , , and , and outputs the sets and with probabilty at least .
We prove thm:main in sec:proof; in fact, we prove a slightly more general result (thm:sdpintegral-general).
We also show that if the instances satisfy a few weaker requirements, then we can obtain a constant factor bi-criteria approximation algorithm for computing the balanced vertex expansion.
We study the case when is an arbitrary graph, i.e., it does not have constant spectral gap. Note that this case is captured by setting in our model, since the monotone adversary can create any arbitrary graph on . Our proof also allows us to let the graph induced on be an arbitrary graph. Again, this is captured by setting in our model, since the monotone adversary can create any arbitrary graph on . We show that we can use the underlying random bipartite graph between and to obtain a constant factor bi-criteria approximation algorithm for computing the balanced vertex expansion in this case.
There exist universal constants , satisfying the following: there exists a polynomial time algorithm which takes a graph generated from VBM, where and , and outputs with probability at least , a set satisfying and .
Next, we study the case where the edges between and are arbitrary, but is large. As in the previous case, our proof allows the graph induced on to be an arbitrary graph. Again, as before, this case is captured by setting . In this case, we show that for certain ranges of , we can obtain a constant factor bi-criteria approximation algorithm for computing the balanced vertex expansion.
There exist universal constants satisfying the following: there exists a polynomial time algorithm which takes a graph generated from VBM, where , and , and outputs a set satisfying and .
In sec:vbmp, we prove a stronger result: it suffices for to contain a subgraph on vertices having spectral gap at least , to obtain a constant factor bi-criteria approximation algorithm for computing the balanced vertex expansion (thm:vbmp-main).
1.3 Related Work
Stochastic Block Models.
Closely related to the vertex expansion of a graph is the notion of edge expansion which is defined as follows.
For a weighted graph , with non-negative edge weights , the edge expansion of a non-empty set is defined as
where and . The edge expansion of the graph is defined as .
The Stochastic Block Model (we will refer to it as the edge expansion stochastic block model to differentiate it from our block model) is a randomized model for instances that are generated as follows. A set of vertices is arbitrarily partitioned into sets of equal sizes. Between each pair of vertices in , an edge is added independently with probability , and between each pair of vertices in , an edge is added independently with probability (typically ).
Starting with work of Holland et. al.[HLL83], the works of Boppana [Bop87], who gave a spectral algorithm, and of Jerrum and Sorkin [JS98], who gave a metropolis algorithm, contributed significantly to the study of stochastic block models. One of the break through works in the study of SBMs is the work of McSherry [McS01], who gave a simple spectral algorithm for a certain range of parameters. There has been a lot of recent work related to a certain conjecture regarding SBMs, which stated the regime of parameters for which it was possible to detect the presence of communities. Works due to [MNS14, MNS15, MNS17, Mas14] have contributed to proving various aspects of the conjecture. In a recent work, Abbe et. al.[ABH16] showed that the natural SDP relaxation for balanced edge expansion is integral when there is a sufficient gap between and , and ; Mossel et. al.[MNS17] gave an algorithm for a larger regime of parameters which was not based on semidefinite programming. More general SBMs have been studied by Abbe and Sandon [AS15a, AS15b, AS17], Aggarwal et. al.[ABKK15], etc.
Kim et. al.[KBG17]
studied a version of SBM for hypergraphs, and gave algorithms for it based on studying a certain “adjacency tensor”, the analog of the adjacency matrix for hypergraphs. They also study the sum-of-squares algorithms for this model.[LM14] gave a reduction from vertex expansion problems to hypergraph expansion problems. We note that applying this reduction to the instances from our models does not give the model studied by [KBG17]: this reduction will only introduce hyperedges between the sets corresponding to and , whereas the model studied by [KBG17] adds random hyperedges between and . Moreover, many parts of a graph from our model are adversarially chosen.
Semi-random models for edge expansion problems.
Monotone adversarial errors in SBMs are the arbitrary addition of edges between pairs of vertices within (resp. ), and the arbitrary deletion of existing edges between and . Feige and Kilian [FK01] gave an algorithm for the edge expansion model with monotone adversarial errors when the gap between and is sufficiently large. Guedon and Vershynin [GV16] gave an algorithm based on semidefinite programming for partially recovering the communities for certain ranges of parameters. Moitra et. al.[MPW16] gave algorithms (based on semidefinite programming) and lower bounds for partial recovery in the stochastic block model with a monotone adversary. Makarychev et. al.[MMV16] gave an algorithm for partial recovery for the stochastic block model with a monotone adversarial errors and a small number of arbitrary errors (i.e. non-monotone errors).
Makarychev et. al.[MMV12, MMV14] studied some semi-random models of instances for edge expansion problems. In particular, [MMV12] studied a model analogous to VBMfor edge expansion problems; they showed that if the number of edges crossing is , and if there is a set of edges such that is a regular graph having spectral expansion at least , then there is an algorithm to recover a balanced cut of edge expansion . The proof of thm:vbmp and that of the corresponding result in [MMV12] both proceed by using the expansion of the underlying subgraph to show that an
sized subset of the SDP vectors lie in a ball of small radius.[MMV12] use this to recover a constant factor bi-criteria approximation to balanced edge expansion; we adapt this approach to vertex expansion to prove thm:vbmp.
The results cited here are only a small sample of the work on the SBMs. Since our model is very different from the edge expansion stochastic block models, we only give a brief survey of the literature here, and we refer the reader to a survey by Abbe [Abb17] for a comprehensive discussion. In general, algorithms for edge expansion problems can not be used for our vertex expansion block model since sparse edge cuts and sparse vertex cuts can be uncorrelated; we give an example to illustrate this fact in app:edgeexpansionbad. In particular, the action of the monotone adversary in VBM rules out the use of edge-expansion based algorithms for detecting and .
There has been some work in investigating vertex expansion (balanced and non-balanced) the worst-case setting. Bobkov et. al. [BHT00] gave a Cheeger-type inequality for vertex expansion, where a parameter plays a role analogous to the use of the second eigenvalue in the edge-expansion variant. Feige et. al.[FHL08] gave a -approximation algorithm for the problem of computing the vertex expansion of graphs. Louis et. al.[LRV13] gave an SDP rounding based algorithm that computes a set having vertex expansion at most , where is the maximum vertex degree; they also showed a matching hardness result based on the Small-set expansion hypothesis. Louis and Makarychev [LM14] gave a bi-criteria approximation for Small-set vertex expansion, a problem related to vertex expansion. Chan et. al. [CLTZ18] studied various parameters related to hypergraphs, including parameters related to hypergraph expansion; they showed that many of their results extend to the corresponding vertex expansion analogues on graphs.
[LR14] studied a model of instances for vertex expansion similar to ours. In their model, the adversary partitions the vertex set into two equal sized sets , and chooses a subset (resp. ) of (resp. ) of size at most . Next, the adversary chooses an arbitrary subset of pairs of vertices in (resp. ) to form edges such that graph induced on (resp. ) is an edge expander. The adversary chooses an arbitrary subset of the pairs of vertices in to form edges. [LR14] give an SDP rounding based algorithm to compute a set having vertex expansion ; we reproduce their proof in app:lr14.
1.4 SDP Relaxation
We use the SDP relaxation for (sdp:primal), this SDP is very similar to that of [LRV13]. We give the dual of this SDP in sdp:dual (we show how to compute the dual SDP in app:dual).
Sdp 1.6 (Primal).
Sdp 1.7 (Dual).
Here denotes the all-ones vector, and denotes the Laplacian matrix of graph weight by the matrix , i.e.
First, let us see why sdp:primal is a relaxation for . Let be the set corresponding to , and let be a vector such is equal to if and otherwise. Note that since , we have . It is easy to verify that and is a feasible solution for sdp:primal, and that . Therefore, . and therefore, sdp:primal is a relaxation for . Henceforth, we will use to be the indicator vector of a set , i.e., is equal to if and otherwise. We prove the following theorem about sdp:primal.
For the regime of parameters stated in thm:main, and for each , for the set defined in VBM, is the unique optimal solution to sdp:primal with probabilty at least .
thm:sdpintegral gives an algorithm to compute the matrix . By factorizing this matrix, one can obtain the vector , using which the set can be computed. Therefore, thm:sdpintegral implies thm:main.
In sec:const-factor-approx, we give a rounding algorithm for sdp:primal, which we use to prove thm:vbmp and thm:vbmlambda.
1.5 Proof Overview
It is easy to verify that is a feasible solution to sdp:primal. Our goal will be to construct a dual solution (i.e. a feasible solution to sdp:dual) which satisfies two properties,
The cost of this solution should be same as the cost of this primal solution .
The matrix should have rank .
Using strong duality, (1) will suffice to ensure that is an optimal solution of the primal SDP . To show that this is the unique primal optimal solution, we will use the complementary slackness conditions which state that
For the sake of completeness, we give a proof of this in app:dual. Since, will have rank , this will imply that all primal optimal solutions must have rank at most , or in other words, there is a unique primal optimal solution (see lem:primal-integ).
While the approach of using complementary slackness conditions for proving the integrality of the SDP relaxation has been studied for similar problems before ([CO07, ABBS14, ABH16, HWX16, ABKK17]), there is no known generic way of implementing this approach to any given problem. Usually the challenging part in implementing this approach is in constructing an appropriate dual solution, and that, like in most of the works cited above, forms the core of our proof.
We give an outline of how we construct our dual solutions. We begin by setting the value for each edge added by the monotone adversary to , thus our proof can be viewed as saying that sdp:primal “ignores” all those edges. For the sake of simplicity, let us consider the case when the bipartite graph between and is a -regular graph. We set if and if . Thus, if we can choose such that this choice of is a feasible solution, then this will ensure that the cost of this dual solution, and the cost of the primal solution are both equal to , thereby fullfilling our first requirement.
If is a rank one matrix, and is a rank matrix, then eq:cs implies that
is an eigenvector ofwith eigenvalue . This fact will be extremely useful in setting the values for the edges in the bipartite graph between and (lem:setting-B). Now, we only have to choose the values for the edges fully contained in (resp. ). We first prove the following lemma which will help us to choose the values.
Lemma 1.9 (Informal statement of lem:suff-cond-lemma).
There exists a constant such that it suffices to choose satisfying
The proof of this lemma follows by carefully choosing the value of , and by exploiting the fact that is an eigenvector of with eigenvalue . Proving the condition in lem:suff-cond-informal can be viewed as the problem of choosing capacities for the edges to support the multicommodity flow where each vertex wants to send amount of flow to each . This idea can work when (resp. ) is a sufficiently dense graph, but does not work when (resp. ) is sparse (rem:flow-lower-bound). Our second idea is to use the edge expansion properties of the underlying spanning subgraph. For a -regular edge expander having the second smallest normalized Laplacian eigenvalue , we get that . Since is a Laplacian matrix, we get that
Now, since and contain an almost regular edge expander as a spanning subgraph, we can adapt the expander argument to this setting and obtain some lower bound on this quantity. This strategy can work in some special cases, but fails in general (rem:expansion-lower-bound). Our proof shows that the desired lower bound in lem:suff-cond-informal can be obtained using a careful combination of these two ideas, in addition to exploiting the various properties of the random graph between and (resp. and ).
1.5.2 thm:vbmlambda and thm:vbmp
We first solve sdp:primal and obtain a matrix such that . Therefore, can be factorized into for some matrix . Let denote the columns of this matrix . We give an algorithm (see sec:const-factor-approx) to “round” these vectors into a set satisfying the guarantees in the theorem. As in the previous case, we show that we can “ignore” all the edges added by the monotone adversary, and only focus on the edges added in step:rand, 3, 4 in def:vbm.
A well known fact for edge expander graphs having roughly equal vertex degrees is that if the value of averaged over all edges in the graph is small, then the value of averaged over all pairs of vertices in the graph is also small. In the proof of thm:vbmp, we use the expansion properties of the sized subset of coupled with this fact to show that an sized subset of the vectors must lie in a ball of small diameter; this step is similar to the corresponding step of [MMV12]. We use this to construct an embedding of the graph onto a line, and recover a cut from this embedding using an algorithm of [LRV13]; this step can be viewed as adapting the corresponding step of [MMV12] to vertex expansion.
In the case when , we show that the lopsided random bipartite graph between and is an edge expander w.h.p. However, this graph is not close to being regular; the degrees of the vertices in would be much higher than the degrees of the verticies in . Therefore, we can not directly use the strategy employed in the previous case. But we show that we can use the fact that the measure of under the stationary distribution of the random bipartite graph between and is , and that the vertices in have roughly equal vertex degrees, to show that averaged over all pairs of vertices is small. From here, we proceed as in the previous case.
obsObservation LABEL:#1 fctFact LABEL:#1
We denote graphs by , where the vertex set is identified with . The vertices are indexed by , or, if belonging to the specific subset () in the VBM model, we use (resp. ) for clarity. The optimal value of the vertex expansion on an instance is denoted by , and the value attained by the algorithm is denoted by . The value of the primal SDP relaxation for vertex expansion on is denoted by , and the value of the dual by . For any , we denote the induced subgraph on by . Given and , define , and . We denote , and . For a subgraph of , the degree of within will correspondingly be .
Given a graph with a weight on its edges, we define the weighted degree of a vertex as . The (un-normalized) Laplacian of a graph with a weight function on its edges is given by , where and . Similar to the unweighted degrees, for any , we define . We will call a graph as close-to-regular or almost regular, if the ratio is at most some constant.
Typically, for a vector , its -th component is denoted by , or in rare cases for clarity, by . The Hadamard product of two matrices is denoted by . As an exception, when we are dealing with vectors associated by the SDP solutions to the vertices of a graph, we exclusively use to be the vector associated with vertex .
We note that for any vector , we have . For a , we denote
In our proofs, following sdp:dual, we will be assigning directed weights (or capacities) to edges , and use to denote the Laplacian of the graph with weights on the edges. Often, when clear from context, we drop the argument for clarity.
Probability distributions will defined over some finite set
. Given a random variable, its expectation is denoted by
. When the distribution is not specified explicitly, it is assumed to be the uniform distribution on, and expectations with respect to the uniform distribution are written as .
We say that an event related to some graph occurs with high probability, if , where is the number of vertices in .
Given an undirected graph , denote the stationary distribution over the vertices by , defined as . Given the normalized Laplacian , the spectral gap of denoted by , is the second-smallest eigenvalue of . Spectral expanders are a family of graphs with at least some constant (independent of the number of vertices in ).
As in the introduction, we use for any to denote the vector in having entries , if , and otherwise.
2 Exact Recovery for Vbm
2.1 A sufficient condition
In order to prove thm:sdpintegral, we first prove the following lemma, which outlines a sufficient condition for integrality of the primal optimal SDP solution.
For a VBM instance, if there exists a that satisfies:
For every and , we have and ,
where , then sdp:primal has as defined in thm:sdpintegral as its unique optimal solution.
While conditions (a) and (b) are explicitly part of the sdp:dual constraints, the remaining conditions (c) and (d) together ensure that we can extend to a feasible dual solution , that satisfies the positive-semidefiniteness constraint and is optimal. As is a spanning forest on the bipartite subgraph on , the weights mentioned in condition (d) are always well-defined.
Proof (of lem:suff-cond-lemma).
We begin by noting a simple consequence of the complementary slackness conditions. We drop the argument from as it is clear from context.
Let be constructed using an optimal dual solution . The primal optimal solution is integral and unique if is a unique eigenvector of with eigenvalue .
Suppose one of the optimal solutions of rank . Consider the spectral decomposition of
where are the eigenvalues, and are the eigenvectors of . Since complementary slackness (See app:dual) implies that , we get . Since , we should have , for each , meaning every is a zero eigenvector of . This is a contradiction for , since has a unique eigenvector. Thus, is rank , and by the assumption, it is a linear multiple of . By the constraints in sdp:primal saying , we get that . ∎
It is thus sufficient to prove that the conditions in lem:suff-cond-lemma imply that we can use the given to come up with a and , such that is feasible, and is a unique eigenvector of with eigenvalue . We first find a (depending on ) that yields a dual objective value of exactly , and ensures that is an eigenvector with eigenvalue . Recall that is the weighted degree of into .
For every and , we have . Further, if and .
Consider an . Condition (d) in lem:suff-cond-lemma already gives us that . For any other , we should have , as are all non-negative, and . A similar argument holds, if . ∎
Fix some partial candidate dual solution . Consider the diagonal matrix given by:
Then is an eigenvector of with eigenvalue . Furthermore, if is feasible for this and satisfies:
then the dual variable assignment is optimal, with objective value .
To prove the first part, we show that . To see this, fix some (a similar argument holds for ), and consider that:
The first equality follows from the fact that . The second equality is due to the fact that within or , is a constant, and hence edges within these do not contribute to the sum. Thus, we only need to look at edges of across the bipartite graph on . The final step used the fact that since we are within the bipartite subgraph on , we have for . The above implies that . By the definition of , we infer that it has as an eigenvector.
In order to prove the second part of the lemma, let be a feasible solution pair for the above that satisfies the given conditions. Then, we have that the dual objective value is:
Above, (a) follows from the definition of and , (b) follows from the fact that every such appears exactly twice in the previous sum. Finally, follows from the fact that the weights from to any are from obs:weights-yts-zero, and the dual SDP (sdp:dual) sets the sum of the weights out of every node to be equal to . Since is a spanning subgraph, every node in contributes exactly to the sum.
Since the primal has a feasible integral solution of value , it follows that such a feasible dual solution is indeed optimal. ∎
As obs:weights-yts-zero shows that the conditions in lem:suff-cond-lemma cover the conditions required on a candidate in lem:setting-B, it now remains to show that a that obeys the preconditions in lem:suff-cond-lemma satisfies:
We first simplify the RHS in the above equation. Condition in lem:suff-cond-lemma sets for every edge between and in (and similarly for ). We use the setting for specified by lem:setting-B. This gives us that for every , , since every edge incident on can have weight at most . Thus, we have:
To tackle the LHS, we will use the following fact:
If is a symmetric matrix with eigenvector having eigenvalue , then:
The forward implication is straightforward, and it in fact holds for all . For the reverse implication: if has a negative eigenvalue, then clearly, adding changes only the eigenvalue corresponding to . Thus, will continue to have a negative eigenvalue. ∎
We will use this fact with , and ; by our setting for , is an eigenvector of with eigenvalue . We first state and prove some lemmas which we use to prove lem:suff-cond-lemma.
Let . For any , and , we have:
Substituting into the LHS gives us: