An spectral sparsifier [ST04] of a weighted graph is a weighted graph such that for every we have the multiplicative guarantee:
There have been many papers on efficiently constructing sparsifiers with few edges (e.g. [SS11, FHHP11, AZLO15, LS17]). The best known sparsity-approximation tradeoff is achieved in [BSS12], who showed that every graph on vertices has an -spectral sparsifier with edges (and this is also the best known upper bound for -cut sparsification). Since every weighted edge can be stored using bits555After discretizing the weights to, say, precision, which introduces negligible error., storing such a sparsifier requires bits.
The recent work [ACK16] studied the question of whether it is possible to use substantially fewer bits if one simply wants a data structure (not necessarily a graph) which answers cut and quadratic form queries. We will use the following definition, which is inspired by the one in [JS17].
An spectral sketch of a graph is a function such that for every :
An cut sketch is a function such that (2) holds for all .
A sketching scheme is a deterministic map from graphs on
vertices to (-spectral or -cut) sketches
, along with specified procedures for storing the
functions as bit strings and for evaluating any given on a query
666 We will not be concerned with the details of these procedures since we are only interested in the space used by the sketches and
not computational parameters such as query time / success probability
of randomized schemes.
We will not be concerned with the details of these procedures since we are only interested in the space used by the sketches and not computational parameters such as query time / success probability of randomized schemes.. The number of bits required to store is called its size.
In [ACK16] it was shown that any -cut sketch must use bits in the worst case, leaving a logarithmic gap between the best known upper and lower bounds. In this paper, we close this gap by showing that any -cut sketching scheme must in fact use bits whenever (Theorem 3.2), which means that it is not in general possible to obtain any asymptotic savings by considering sketches which are not graphs. We also give a lowerbound for -spectral sketching with a simpler proof and slightly better constants (Theorem 2.1).
1.1 Related Work
The paper [ACK16] also studied the problem of producing a “for each” sketching algorithm, which has the property that for any particular query , approximates with constant probability. They showed that in this weaker model it is possible to obtain cut sketches of size and -spectral sketches of size . The latter bound was recently improved to and generalized to include queries to the pseudoinverse in [JS17].
In contrast, in this paper we study only the “for all” model, in which the sketch is required to work for all queries simultaneously.
Our proof is based on the following rigidity phenomenon: there is a constant such that if two regular graphs -cut approximate each other then they must overlap in a constant fraction of edges (Lemmas 2.2 and 3.1). This allows us to argue that below this approximation threshold any sketch encodes a constant fraction of the information in the graph, and so any sketch must use bits.
Interestingly, this phenomenon is very sensitive to the value of the constant — in particular, a well-known theorem of Friedman [Fri03]
says that the eigenvalues of the Laplacian of a randomregular graph are contained in the range with high probability, which implies that almost every regular graph has a -spectral sketch of constant size, namely the complete graph.
Note that according to the Alon-Boppana bound [Nil91] it is not possible to approximate by a regular graph with error less than . Thus, our result can be interpreted as saying that a regular graph can only be approximated by nearby graphs when the error is substantially below the Alon-Boppana bound.
We remark that the proof of [ACK16] was based on showing a bit lower bound for a constant sized graph by reducing to the Gap-Hamming problem in communication complexity, and then concatenating instances of this problem to obtain an instance of size . This method is thereby inherently incapable of recovering the logarithmic factor that we obtain (in contrast, we work with random regular graphs).
1.3 Notation and Organization
We will denote -spectral approximation as
and cut approximation as
We will use and to denote binary and natural logarithms.
We prove our lower bound for spectral sketching in Section 2, and the lower bound for cut sketching in Section 3. Although the latter is logically stronger than the former, we have chosen to present the spectral case first because it is extremely simple and introduces the conceptual ideas of the proof.
We would like to thank MSRI and the Simons Institute for the Theory of Computing, where this work was carried out during the “Bridging Discrete and Continuous Optimization” program.
2 Lower Bound for Spectral Sketches
In this section, we prove the following theorem.
For any , any -spectral sketching scheme for graphs with vertices must use at least bits in the worst case.
The main ingredient is the following lemma.
Lemma 2.2 (Rigidity of Spectral Approximation).
Suppose and are simple regular graphs such that
Then and must have at least edges in common.
Since and are regular, we have and . Thus the hypothesis implies that:
which means . Passing to the Frobenius norm, we find that
The matrix has entries in , with exactly two nonzero entries for every edge in , so the left hand side is equal to
since . Rearranging yields the desired claim. ∎
Note that the above lemma is vacuous for but indicates that and must share a fraction of their edges for below this threshold.
For any function and :
where denotes the set of all regular graphs on vertices.
If does not approximate any regular graph then we are done. Otherwise, let be the lexicographically (in some pre-determined ordering on regular graphs) first graph which -approximates. Suppose is another graph that approximates. Notice that by applying (2) twice, we have that for every :
so -spectrally approximates . By Lemma 2.2, and must share
edges. Thus, can be encoded by specifying:
Which edges of occur in . This is a subset of the edges of , which requires at most bits.
The remaining edges of . Each edge requires at most bits to specify, so the number of bits needed is at most .
Thus, the total number of bits required is at most
as desired. ∎
([Wor99]Cor. 2.4) For , the number of regular graphs on vertices is:
Proof of Theorem 2.1.
Let be the number of distinct sketches produced by and let . By Lemma 2.3, the binary logarithm of the number of regular graphs -spectrally approximated by any single sketch is at most
On the other hand by Theorem 2.4, since we have
Since every regular graph receives a sketch, we must have
The proof above actually shows that any -spectral sketching scheme for regular graphs on vertices must use at least bits on average, since the same proof goes through if we only insist that the sketches work for most graphs.
The result of [BSS12] produces -spectral sparsifiers with edges, which yield -spectral sketches with bits by discretizing the edge weights up to error, so the bound above is tight up to a factor of . We have not made any attempt to optimize the constants.
3 Lower Bound for Cut Sketches
In this section we prove Theorem 3.2. The new ingredient is a rigidity lemma for cuts, which may be seen as a discrete analogue of Lemma 2.2. The lemma holds for bipartite graphs and is proven using a Goemans-Williamson [GW95] style rounding argument.
Lemma 3.1 (Rigidity of Cut Approximation).
Suppose and are simple regular bipartite graphs with the same bipartition , such that
Then and must have at least edges in common.
We will show the contrapositive. Assume
for some . To show that (4
) does not hold, it is sufficient to exhibit a vectorsuch that
where , since both graphs are regular and the latter inequality follows from . To find such an we will first construct vectors such that
and then use hyperplane rounding to find scalarswhich satisfy:
Let be the columns of
and note that since and every vertex is incident with at most edges in each of and . The relevant inner products of the are easy to understand:
where in the latter two cases we have used the fact that and cannot have any common neighbors because they lie on different sides of the bipartition. Letting , we therefore have
by our choice of . Let be a random unit vector and let
We denote the angle between vectors as . We recall that since and have unit length. It follows that the probability that is equal to and the probability of is equal to . Thus,
by (7), as desired.
The analysis of the rounding scheme above can be improved from to the Goemans-Williamson constant , but we have chosen not to do so for simplicity.
For any , any -cut sketching scheme for graphs with vertices must use at least bits in the worst case.
Assume is divisible by (add a constant if this is not the case). Let and let be the set of bipartite graphs on vertices with respect to a fixed bipartition. We proceed as in the proof of Theorem 2.1. Let be any function which is an cut sketch for some graph . Arguing as in Lemma 2.3, any other graph which has the same sketch must -cut approximate , so by Lemma 3.1, any such must have at most edges which are not present in . Thus, the encoding length of such is at most
bits, by our choice of , so any particular can only be an -cut sketch for graphs in
On the other hand, is quite large. Recall that the bipartite double cover of a graph on vertices is the graph on
vertices obtained by taking its tensor product with, and two distinct graphs must have distinct double covers. Thus, by (3) we have
Thus, if is the number of distinct sketches produced by , we must have
as desired. ∎
- [ACK16] Alexandr Andoni, Jiecao Chen, Robert Krauthgamer, Bo Qin, David P Woodruff, and Qin Zhang. On sketching quadratic forms. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science, pages 311–319. ACM, 2016.
- [AZLO15] Zeyuan Allen-Zhu, Zhenyu Liao, and Lorenzo Orecchia. Spectral sparsification and regret minimization beyond matrix multiplicative updates. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing, pages 237–245. ACM, 2015.
- [BK96] András A Benczúr and David R Karger. Approximating st minimum cuts in õ (n 2) time. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pages 47–55. ACM, 1996.
- [BSS12] Joshua Batson, Daniel A Spielman, and Nikhil Srivastava. Twice-ramanujan sparsifiers. SIAM Journal on Computing, 41(6):1704–1721, 2012.
- [FHHP11] Wai Shing Fung, Ramesh Hariharan, Nicholas JA Harvey, and Debmalya Panigrahi. A general framework for graph sparsification. In Proceedings of the forty-third annual ACM symposium on Theory of computing, pages 71–80. ACM, 2011.
- [Fri03] Joel Friedman. A proof of alon’s second eigenvalue conjecture. In Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, pages 720–724. ACM, 2003.
- [GW95] Michel X Goemans and David P Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), 42(6):1115–1145, 1995.
- [JS17] Arun Jambulapati and Aaron Sidford. Efficient spectral sketches for the laplacian and its pseudoinverse. arXiv preprint arXiv:1711.00571, 2017.
- [LS17] Yin Tat Lee and He Sun. An sdp-based algorithm for linear-sized spectral sparsification. arXiv preprint arXiv:1702.08415, 2017.
- [Nil91] Alon Nilli. On the second eigenvalue of a graph. Discrete Mathematics, 91(2):207–210, 1991.
- [SS11] Daniel A Spielman and Nikhil Srivastava. Graph sparsification by effective resistances. SIAM Journal on Computing, 40(6):1913–1926, 2011.
- [ST04] Daniel A Spielman and Shang-Hua Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In Proceedings of the thirty-sixth annual ACM symposium on Theory of computing, pages 81–90. ACM, 2004.
- [Wor99] Nicholas C Wormald. Models of random regular graphs. London Mathematical Society Lecture Note Series, pages 239–298, 1999.