High-Dimensional Expanders from Expanders

07/24/2019
by   Siqi Liu, et al.
berkeley college
0

We present an elementary way to transform an expander graph into a simplicial complex where all high order random walks have a constant spectral gap, i.e., they converge rapidly to the stationary distribution. As an upshot, we obtain new constructions, as well as a natural probabilistic model to sample constant degree high-dimensional expanders. In particular, we show that given an expander graph G, adding self loops to G and taking the tensor product of the modified graph with a high-dimensional expander produces a new high-dimensional expander. Our proof of rapid mixing of high order random walks is based on the decomposable Markov chains framework introduced by Jerrum et al.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

05/19/2021

Improved Product-Based High-Dimensional Expanders

High-dimensional expanders generalize the notion of expander graphs to h...
06/07/2021

A Matrix Trickle-Down Theorem on Simplicial Complexes and Applications to Sampling Colorings

We show that the natural Glauber dynamics mixes rapidly and generates a ...
11/09/2020

High Dimensional Expanders: Random Walks, Pseudorandomness, and Unique Games

Higher order random walks (HD-walks) on high dimensional expanders have ...
02/01/2017

High Order Stochastic Graphlet Embedding for Graph-Based Pattern Recognition

Graph-based methods are known to be successful for pattern description a...
11/05/2018

Log-Concave Polynomials II: High-Dimensional Walks and an FPRAS for Counting Bases of a Matroid

We use recent developments in the area of high dimensional expanders and...
01/02/2020

Spectral Independence in High-Dimensional Expanders and Applications to the Hardcore Model

We say a probability distribution μ is spectrally independent if an asso...
05/03/2020

Locally testable codes via high-dimensional expanders

Locally testable codes (LTC) are error-correcting codes that have a loca...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Expander graphs are graphs which are sparse, yet well-connected. They play important roles in applications such as the construction of pseudorandom generators and error-correcting codes [SS94]. Motivated by both purely theoretical questions, such as the topological overlapping problem, and applications in computer science, such as PCPs, a generalization of expansion to high dimensional complexes has recently emerged. We work with –dimensional complexes, which not only have vertices and edges, but also hyperedges of vertices, for any . Whereas in the one-dimensional world of graphs, the properties of edge expansion, spectral expansion and rapid mixing of random walks are equivalent, their generalization to Several different characterizations of “expansion” have been developed for these high–dimensional complexes. In particular, the high-dimensional extension of spectral expansion is simple to state, and implies rapid mixing of high order walks [KO17] and agreement expanders [DK17].

We construct bounded-degree high–dimensional expanders of all constant–sized dimensions, where the high order random walks have a constant spectral gap, and thus mix rapidly. We base our HDX’s from existing -regular one-dimensional constructions, which can be sampled readily from the space of all -regular graphs. This endows a natural distribution from which we can sample HDX’s of our construction as well.

One sufficient, but not necessary criterion that implies rapid mixing is spectral, which comes from the graph theoretic notion below.

[Informal] A –dimensional –spectral expander is a –dimensional simplicial complex (i.e. a hypergraph whose faces satisfy downward closure) such that

  • (Global Expansion) The vertices and edges (sets of two vertices) of the complex constitute a –spectral expander graph,

  • (Local Expansion) For every hyperedge of size in the hypergraph, the vertices and edges in the ”neighborhood” of also constitute a –spectral expander. (The precise definition of ”neighborhood” will be discussed later.)

Most known constructions of bounded-degree high–dimensional spectral expanders are heavily algebraic, rather than combinatorial or randomized. In contrast, there are a wealth of different constructions for bounded-degree (one-dimensional) expander graphs [HLW06]. Some of these are also algebraic, such as the famous LPS construction of Ramanujan graphs [LPS88]

, but there are also many simple, probabilistic constructions of expanders. In particular, Friedman’s Theorem says that with high probability, random

-regular graphs are excellent expanders [Fri03].

Unfortunately, random –dimensional hypergraphs with low degrees are not –dimensional expander graphs. For a hypergraph with vertices, we need a roughly -degree Erdos-Renyi graph to make the neighborhood of every hyperedge of size to be connected with high probability. While random low degree hypergraphs are not high–dimensional expanders, our construction provides simple probabilistic high–dimensional expanders of all dimensions.

1.1 Technical Overview

Construction.

We construct an –dimensional simplicial complex on vertices, from a graph of vertices and a (small) –dimensional complete simplicial complex on vertices. To construct , we replace each vertex of with a copy of which we denote . Denote the copy of a vertex in by . The faces of are chosen in the following way: for every face in , add to the graph, where for some edge in , the vertices are one of the endpoints of . The main punchline of our work is that when is a (triangle-free) expander graph, the high order random walks on mix rapidly. Specifically, we prove: [Main theorem, informal version of thm:mixing-main] Suppose is a triangle-free expander graph with two-sided spectral gap . For every such that , there is a constant depending on , but independent of such that the Markov transition matrix for the up-down walk on the -faces of has two-sided spectral gap .

First attempt at proving rapid mixing of high order random walks.

[KM16], which introduced the notions of up-down and down-up random walks, and subsequent works [DK17, KO17, KO19, ALGV19] developed and followed the “local-to-global paradigm” to prove rapid mixing of high order random walks. In particular, each of these works would:

  1. Establish that all the links of a relevant simplicial complex have “small” second eigenvalue.

  2. Prove or cite a statement about how rapid mixing follows from small second eigenvalues of links (such as thm:informal-KO17).

Then, step:links-small-eigen and step:local-exp-to-rapid-mix together would imply that the up-down and down-up random walks on the simplicial complexes they cared about mixed rapidly. This immediately motivates first bounding the second eigenvalue of the links of our construction, and applying the quantitatively strongest known version of the type of theorem alluded to in step:local-exp-to-rapid-mix. Thus, in sec:local-exp we analyze the second eigenvalue of all links of and prove: [Informal version of thm:local-exp-main] The two-sided spectral gap of every link in is bounded by approximately . And the ‘quantitatively strongest’ known “local-to-global” theorem is [Informal statement of [KO17, Theorem 5]] If the second eigenvalue of every link of a simplicial complex is bounded by , then the up-down walk on -faces of , satisfies:

Observe that the upper bound on the second eigenvalue of all links must be strictly less than to conclude any meaningful bounds on the mixing time of the up-down random walk. Thus, unfortunately, thm:informal-local-exp in conjunction with thm:informal-KO17 fails to establish rapid mixing.

Hence, we depart from the local-to-global paradigm and draw on alternate techniques.

Decomposing Markov Chains.

Each -face of is either completely contained in a cluster for a single vertex in , or straddles two clusters corresponding to vertices connected by an edge, i.e., is contained in . Consider performing an up-down random walk on the space of -faces of (henceforth ). If we record the single cluster or pair of clusters containing the -face the random walk visits at each timestep, it would resemble:

In the above illustration of a random walk, let us restrict our attention to the segment of the walk where the -faces are all contained in, say, the pair of clusters . Intuitively, we expect the random walk restricted to those -faces to mix rapidly and also exit the set quickly by virtue of the state space being constant-sized. In particular, if we keep the random walk running for steps for some large constant , it would seem that the number of “exit events”111Transitions like , , and so on. is roughly for some other constant . The sequence of “exit events” can be viewed as a random walk on the space of edges and vertices of , and since there are many steps in this walk, the expansion properties of tell us that the location of the random walk after steps is distributed according to a relevant stationary distribution. In light of these intuitive observations of rapidly mixing in the walks within cluster pairs and also rapidly mixing in a walk on the space of cluster pairs, one would hope that the up-down walk on -faces mixes rapidly.

This hope is indeed fulfilled and is made concrete in a framework of Jerrum et al. [JST04]. In their framework, there is a Markov chain on state space . They show that if can be partitioned into such that the chain “restricted” (for some formal notion of restricted) to each , and an appropriately defined “macro-chain” (where each partition is a state) each have a constant spectral gap, then the original Markov chain has a constant spectral gap as well. Our proof of the fact that has a constant spectral gap utilizes this result of [JST04]. This framework of decomposable Markov chains is detailed in sec:decompose, and the analysis of the spectral gap of the down-up random walk222Which is actually equivalent to proving a spectral gap on the up-down random walk but is more technically convenient. See fact:ud-du-same-spectra. is in sec:spec-gap-HO.

1.2 Related Work

While high–dimensional expanders have been of relatively recent interest, already many different (non-equivalent) notions of high–dimensional expansion have emerged, for a variety of different applications.

The earliest notions of high–dimensional expansion were topological. In this vein of work, [LM06, Gro10] introduced coboundary expansion, [EK16] defined cosystolic expansion, and [EK16, KKL14] defined skeleton expansion. To our knowledge, most existing constructions of these types of expanders rely on the Ramanujan complex. We refer the reader to a survey by Lubotzky [Lub17] for more details on these alternate notions of high dimensional expansion and their uses.

To describe notions of high dimensional expansion that are relevant to computer scientists, we need to first highlight a key property of (one-dimensional) expander graphs–that random walks on them mix rapidly to their stationary distribution. The notion of a random walk on graphs was generalized to simplicial complexes in the work of Kaufman and Mass [KM16] to the “up-down” and “down-up” random walks, whose states are -faces of a simplicial complex. They were interested in bounded–degree simplicial complexes where the up-down random walk mixed to its stationary distribution rapidly. They then proceed to show that the known construction of Ramanujan complexes from [LSV05] indeed satisfy this property.

A key technical insight in their work that the rapid mixing of up-down random walks follows from certain notions of local spectral expansion, i.e., from sufficiently good two-sided spectral expansion of the underlying graph of every link. A quantitative improvement between the relationship between the two-sided spectral expansion of links and rapid mixing of random walks was made in [DK17], and this improvement was used to construct agreement expanders based on the Ramanujan complex construction. Later, [KO17] showed that one-sided spectral expansion of links actually sufficed to derive rapid mixing of the up-down walk on -faces.

1.2.1 HDX Constructions

Although this combinatorial characterization of high–dimensional expansion is slightly weaker than some of the topological characterizations mentioned above, few constructions are known for bounded degree HDX’s with dimension . Most of these rely on heavy algebra. In contrast, for one-dimensional expander graphs, there are a wealth of different constructions, including ones via graph products and randomized ones. [Fri03] states that even a random -regular graph is an expander with high probability.

The most well-known construction of bounded-degree high–dimensional expanders are the Ramanujan complexes [LSV05]. These require the Bruhat-Tits building, which is a high-dimensional generalization of an infinite regular tree. The underlying graph has degree , where is a prime power satisfying . The links can be described by spherical buildings

, which are complexes derived from subspaces of a vector space, and are excellent expanders.

Dinur and Kaufman showed that given any , and any dimension , the –skeleton of any –dimensional Ramanujan complex is a –dimensional –spectral expander [DK17]. Here, the degree of each vertex is . In other words, they “truncate” the Ramanujan complexes, throwing out all faces of size greater than some number . Their primary motivation was to obtain agreement expanders, which find uses towards PCPs.

Recently, Kaufman and Oppenheim [KO19] present a construction of one–sided high–dimensional expanders, which are coset complexes of elementary matrix groups. The construction guarantees that for any and any dimension , there exists a infinite family of high–dimensional expanders , such that (1) every are –dimensional –one–sided–expander; (2) every ’s 1-skeleton has degree at most ; (3) as goes to infinity the number of vertices in also goes to infinity.

Even more recently, Chapman, Linial, and Peled [CLP18] also provided a combinatorial construction of two-dimensional expanders. They construct an infinite family of -regular graphs, which are -regular graphs whose links with respect to single vertices are -regular. The primary motivation for their construction comes from the theory of PCPs. They prove an Alon-Boppana type bound on for any -regular graph, and construct a family of graphs where this bound is tight. They also build an -regular two-dimensional expander using any non-bipartite graph of sufficiently high girth; they achieve a local expansion only depending on the girth, and the global expansion depending on the spectral gap of . Like ours, their construction also resembles existing graph product constructions of one-dimensional expanders.

2 Preliminaries and Notation

2.1 Spectral Graph Theory

While we can describe our constructions combinatorially, our analysis of both the mixing times of certain walks as well as the local expansion will heavily rely on understanding graph spectra.

For a graph on vertices, we use to denote its normalized adjacency matrix, i.e. the matrix

and write its eigenvalues as

Let to indicate the set . We write for the spectral gap of , which is the quantity . Graphs with are said to be one-sided -expanders.

Most of the graphs we analyze achieve a stronger condition; that the second largest eigenvalue magnitude is not too large. Formally, we write for the -th largest eigenvalue in absolute value. In particular, . The absolute spectral gap of , denoted , is the quantity . Graphs with are two-sided -expanders.

This definition is also easily extended to graphs that have edge weights. If we have a weight function , the normalized adjacency matrix is given by:

In this case, is still ; the rest of the definitions remain the same.

If we consider as a transition matrix for a random walk on , having a large spectral gap and large expansion implies fast mixing (i.e. convergence to a stationary distribution).

2.1.1 Graph Tensors

Our construction can roughly be described as a tensor product, defined below. The tensor product of two graphs and is given by

  1. Vertex set ,

  2. Edge set .

The adjacency matrix is the tensor (Kronecker) product . Due to this structure, . As is the largest eigenvalue of both and , it follows that both

2.1.2 Line Graphs

The line graph, which is an incidence graph on the edges, will be a useful scaffolding on which we decompose our high order walks.

Given a graph , its line graph has vertices , and edges .

The relationship between and is also well understood.

[[Sac66]] If is a graph of degree with vertices and its line graph, then the characteristic polynomials and satisfy

2.2 Markov Chains

Here, we provide a very basic treatment of the Markov chain concepts required for the analysis of our high order walks. We refer to [LP17] for a detailed and thorough treatment of the fundamentals of Markov chains.

A Markov chain is given by states and a transition matrix where is the probability of going to state from state . We may also write this quantity as .

The Markov chain literature often defines as the transition probability , so their is the transpose of ours. However, when dealing with

’s spectrum, we work with column (right) eigenvectors, while this alternate convention uses row (left) eigenvectors, so both conventions yield the same results.

For graph , is the transition matrix for a Markov chain where states are vertices, and each state transitions to its neighbors uniformly at random. We may use “graph” in lieu of “chain” when we want to indicate this particular Markov chain.

A chain can be associated with a digraph on the states. If is strongly connected, we say is irreducible. If it is possible to go from state to state in using exactly steps for any sufficiently large , we say is aperiodic.

We define the edge set of a Markov chain as

For chain , let be the right eigenvalues of .

Let . For any Markov chain, .

Let . If is irreducible and aperiodic, the associated (normalized) eigenvector of is the stationary distribution of , which we denote by . We call the spectral gap of , which we write as .

The next property we introduce is present for every Markov chain we consider. Intuitively, it means that if start at the stationary distribution and run the chain for a sequence of time states, the reverse sequence has the same probability of occurring. The Markov chain is time-reversible if for any integer :

Time reversibility helps us compute stationary distributions via the detailed balance equations. This is especially helpful when there are a huge number of symmetric states. The Markov chain is time-reversible if and only if it satisfies the detailed balance equations: for all ,

Most of the Markov chains we encounter in this work will also have self-loops. If we start with Markov chain and wish to add a uniform self loop probability to each state to get Markov chain , we write as a convex combination of and :

Since this convex combination will appear a few different times throughout this paper, we’ll prove a basic fact about the spectral gap of : For as defined above:

Proof.

Let be any eigenvalue of , with associated eigenvector . Then, is also an eigenvector for for eigenvalue:

To see this, . Thus, the spectrum of is a linear shift and scaling of the spectrum of , and the spectral gap also scales by . ∎

Note that does not scale the way does. In fact, if we perform the above scaling, the smallest eigenvalue of is at least , so

The -mixing time of a Markov chain is the smallest such that for any distribution over the states of ,

where is the stationary distribution of .

For any Markov chain , the -mixing time satisfies:

2.2.1 Decomposing Markov Chains

The Poincaré inequality is a powerful tool for studying convergence of Markov chains. Consider a finite-state time reversible Markov chain whose structure gives rise to natural state-space partition, can be decomposed into a number of restriction chains and a projection chain. [JST04] show that the spectral gap for the original chain can be lower bounded in terms of the spectral gaps for the restriction and projection chains.

We now formally define decomposition of a Markov chain. Consider an ergodic Markov chain on finite state space with transition probability . Let denote its stationary distribution, and let be a partition of the state space into disjoint sets, where .

The projection chain induced by the partition has state space and transitions

The above expression corresponds to the probability of moving from any state in to any state in in the original Markov chain.

For each , the restriction chain induced by has state space and transitions

is the probability of moving from state to state when leaving is not allowed.

Regardless of how we define the projection and restriction chains for a time reversible Markov chain, they all inherit one useful property from the original chain.

Let be a time-reversible Markov chain. Then, for any decomposition of , the projection and restriction chains are also time-reversible.

We ultimately want to study the spectral gap of random walks. Luckily, the original Markov chain’s spectral is related to the restriction and projection chains’ spectral gaps in the following way:

[[JST04, Theorem 1]] Consider a finite-state time-reversible Markov chain decomposed into a projection chain and restriction chains as above. Define to be maximum probability in the Markov chain that some state leaves its partition block,

Suppose the projection chain satisfies a Poincaré inequality with constant , and the restriction chains satisfy inequalities with uniform constant . Then the original Markov chain satisfies a Poincaré inequality with constant

Recall that if satisfies a Poincaré inequality, it is a lower bound on the spectral gap (cf. [LP17]).

2.3 High-Dimensional Expanders

The generalization from expander graphs to hypergraphs (more specifically, simplicial complexes) requires great care. We now formally establish the high dimensional notions of “neighborhood”, “expansion,” and “random walk.”

A simplicial complex is specified by vertex set and a collection of subsets of , known as faces, that satisfy the “downward closure” property: if and , then . Any face of cardinality is called a -face of . We use to denote the subcollection of -faces of . We say has dimension , where .

A -dimensional complex is a graph with vertex set and edge set .

To formally define random walks and Markov chains on a , we need to associate with a weight function . We want our weight function to be balanced, meaning for :

If we restrict ourselves to balanced , it suffices to only define over and propagate the weights downward to the lower order faces.

Let be a balanced weight function on faces of . Define:

Because of the downward closure property, we always have .

The (weighted) -skeleton of is the complex with vertex set and all faces in of cardinality at most , with weights inherited from .

The -skeleton of only contains its vertices (-faces) and edges (-faces). It can be characterized as a graph with edge weights, so we can also compute and .

For for , we associate a particular -dimensional pure complex known as the link of defined below.

If was equipped with weight function , then “inherits” it. We associate with weight function given by . If is balanced, then is also balanced.

In a graph, the link of a vertex is simply its neighborhood.

The global expansion of , denoted , is the expansion of its weighted -skeleton.

The local expansion of , denoted is

In words, it is equal to the expansion of the worst expanding link.

We use to denote the complete -dimensional complex on vertex set , i.e., the pure -dimensional simplicial complex obtained by making the set of -faces equal to all subsets of of size . The -skeleton is then a clique on vertices whose expansion is and the -skeleton of a -link is a clique on vertices, which has expansion . As a result, .

We often use to refer to the adjacency matrix of the -skeleton of , and we may also use to refer to the -th largest eigenvalue of .

Previously, we mentioned that there are several different notions of high dimensional expansion: some geometric or topological, some combinatorial. We now formally define high dimensional spectral expansion, which is a more combinatorial and graph theoretic notion: is a two-sided -local spectral expander if and .

2.3.1 High Order Walks on Simplicial Complexes

Let be a -dimensional simplicial complex and with weight function on the -faces of , for . For each , we can define a natural (periodic) Markov chain on a state space consisting of -faces and -faces of .

  • At a -face , there are exactly faces such that , due to the downward closure property. We transition from to each -face with probability .

  • At a -face , we transition to each -face satisfying with probability . (Note that must be balanced for these transitions to be well-defined.)

Restricting the above chain to only odd or even time steps gives us two new random walks: one entirely on

and one entirely on .

[Down-up walk on -faces of ] = Let be the Markov chain with state space equal to and transition probabilities described by the process above, where there is an implicit transition down to a -face and back up to a -face. Then:

[Up-down walk on -faces of ] Let be the Markov chain with state space equal to and transition probabilities described by the process above, where there is an implicit transition up to a -face and back down to a -face. Then:

In the literature, we also see written as , and written as .

We now present some facts about these high order walks without proof. We refer to [KO17, ALGV19] for proofs of these facts.

The transition matrices for and share the same eigenvalues. The nonzero eigenvalues occur with the same multiplicity. A straightforward but important consequence of this fact is

The Markov chains and have the same stationary distribution on , which is proportional to for each . We will call this distribution .

For the remainder of the paper, we will assume a uniform weight function on , which is useful for applications like sampling bases of a matroid [ALGV19]. When using the uniform weighting scheme, for , there is a natural interpretation of : the fraction of -faces that contain as a subface. (We also note that we will use symbolic variables to represent various weight values, and that it is straightforward to adapt our computations to cases where we have uniform weights over for any .)

3 Local Densification of Expanders

For a graph and -dimensional simplicial complex , we give a way to combine the two to produce a bounded-degree -dimensional complex of constant expansion. First, construct a graph with

  1. vertex set equal to , and

  2. edge set equal to .

is then defined as the -dimensional pure complex whose -faces are all cliques on vertices such that there exists an edge in for which .

Linear algebraically, we can think of this graph construction as adding a self loop to each vertex of and then taking the tensor product with the -skeleton of .

Our construction is , where is equal to , the -dimensional complete complex on some constant vertices, and is a -regular triangle-free expander graph on vertices. We endow with a balanced weight function induced by setting the weights of all -faces to .

As a first step to understanding this construction, we inspect the weights induced on -faces for . Consider a -face . A short calculation reveals that if are all equal, then is equal to and otherwise, is equal to . Henceforth, write and instead of and when is understood from context.

We now list out what we prove about . Most importantly, we show: For every , the Markov transition matrix for down-up (and equivalently up-down) random walks on the -faces satisfies:

We dedicate sec:spec-gap-HO to proving thm:mixing-main. In sec: intro to duwalk, we show the transition probabilities for the down-up walks on the -faces and derive a lower bound on the smallest eigenvalue of (obs:smallest-eigval). In the rest of the section we decompose the random walk Markov chains to obtain an upper bound on the one-sided spectral gap for (thm:down-up-random-walk-spectral-gap).

As an immediate corollary of thm:mixing-main and thm:spec-gap-to-mixing, we get that Let denote the number of -faces in . Then the -mixing time of satisfies:

We note that .

We also derive bounds on the expansion of links of . In particular, as a direct consequence of thm:main and the discussion of the expansion properties of the complete complex in eg:complete-complex, we conclude: We can prove the following bounds on the local and global expansion of :

Suppose is a random -regular (triangle-free) graph and . Then the corresponding (random) simplicial complex , as a consequence of Friedman’s Theorem [Fri03]333Friedman’s theorem says that a random -regular graph, whp, has two-sided spectral gap . Additionally, random graphs are triangle-free with constant probability., with high probability satisfies

Thus, endows a natural distribution over simplicial complexes that gives a high-dimensional expander with high probability.

If is strongly explicit, such as an expander from [RVW02, BATS11], then is also strongly explicit since the tensor product of two strongly explicit graphs is also strongly explicit.

4 Local Expansion

For this entire section, we will mainly work with the complex , so when we use without a subscript, it will be with respect to . Next, fix a face . In order to study the expansion of the -skeleton of , we need to first compute the weights on its 1-faces.

Let , where as before, and . There are several cases we need to consider:

  1. Case 1: .
    Here, , which is proportional to the number of -faces that contain . The face already has vertices, so there are possibilities of . There are choices for , since must equal .

  2. Case 2: .

    1. Case 2(a): and .
      Again, there are possibilities for . Since , we will have choices for , as has neighbors in , and when is not constant on , there are choices for the other value it can take.

    2. Case 2(b): but , and .
      Again, we have possibilities for , but we only have choices for ; the image of must be .

    3. Case 2(c): but , and .
      The analysis is identical to that of Case 2(b)

For simplicity, we’ll assign weights to the elements of as below:

(Here, the and denote “center” and “satellite,” whose meanings will be more natural when discussing when .)

Note that if we choose (so ), we simply get the weights of the -skeleton of itself, which will be useful for computing global expansion.

Let be a triangle-free -regular graph and let be a pure -dimensional simplicial complex. Then

Proof.

Let be the graph obtained by adding self-loops to , with transitions

For large , the self loop probabilities approach , while the others approach .

First, observe that . Thus,

and hence the second largest absolute eigenvalue is no more than , which is simply equal to . This implies that