Spectral Graph Matching and Regularized Quadratic Relaxations II: Erdős-Rényi Graphs and Universality

1 Introduction

Given two (weighted) graphs, graph matching aims at finding a bijection between the vertex sets that maximizes the total edge weight correlation between the two graphs. It reduces to the graph isomorphism problem when the two graphs can be matched perfectly. Let $A$ and $B$ be the (weighted) adjacency matrices of the two graphs on $n$ vertices. Then the graph matching problem can be formulated as solving the following quadratic assignment problem (QAP) [PRW94, BCPP98]:

max Π \in S_{n} ⟨ A, Π B Π^{⊤} ⟩,

(1)

where $S_{n}$ denotes the set of permutation matrices in $R^{n \times n}$ and $⟨ \cdot, \cdot ⟩$ denotes the matrix inner product. The QAP is NP-hard to solve or to approximate within a growing factor [MMS10].

In the companion paper [FMWX19], we proposed a computationally efficient spectral graph matching method, called GRAph Matching by Pairwise eigen-Alignments (GRAMPA). Let us write the spectral decompositions of $A$ and $B$ as

A = \sum i λ_{i} v_{i} v_{i}^{⊤} and B = \sum j μ_{j} w_{j} w_{j}^{⊤} .

(2)

Given a tuning parameter $η > 0$ , GRAMPA first constructs an $n \times n$ similarity matrix¹¹1In [FMWX19], $X$ is defined without the factor $η$ in the numerator. We include $η$ here for convenience in the proof; this does not affect the algorithm as the rounded solution $ˆ Π$ is invariant to rescaling $X$ .

X = \sum i, j \frac{η}{(λ_{i} - μ_{j})^{2} + η^{2}} v_{i} v_{i}^{⊤} J w_{j} w_{j}^{⊤},

(3)

where $J$ is the $n \times n$ all-ones matrix. Then it outputs a permutation matrix $ˆ Π$ by “rounding” $X$ to a permutation matrix, for example, by solving the following linear assignment problem (LAP)

ˆ Π \in argmax Π \in S_{n} ⟨ X, Π ⟩ .

(4)

Let $Π_{*} \in S_{n}$ be the latent true matching, and denote the entries of $A$ and $Π_{*} B Π_{*}^{⊤}$ as $a_{i j}$ and $b_{π_{*} (i) π_{*} (j)}$ . A Gaussian Wigner model is studied in [FMWX19], where ${(a_{i j}, b_{π_{*} (i) π_{*} (j)})}$ are i.i.d. pairs of correlated Gaussian variables such that $b_{π_{*} (i) π_{*} (j)} = a_{i j} + σ z_{i j}$ for a noise level $σ \geq 0$ , and $a_{i j}$ and $z_{i j}$ are independent standard Gaussian. It is shown that GRAMPA exactly recovers the vertex correspondence $Π_{*}$ with high probability when $σ = O (1 / log n)$ . Simulation results in [FMWX19, Section 4.1] further show that the empirical performance of GRAMPA under the Gaussian Wigner model is very similar to that under the Erdős-Rényi model where ${(a_{i j}, b_{π_{*} (i) π_{*} (j)})}$

are i.i.d. pairs of correlated centered Bernoulli random variables, suggesting that the performance of GRAMPA enjoys universality.

In this paper, we prove a universal exact-recovery guarantee for GRAMPA, under a general Wigner matrix model for the weighted adjacency matrix: Let $A = (a_{i j})$

be a symmetric random matrix in

R^{n \times n}

, where the entries

(a_{i j})_{i \leq j}

are independent. Suppose that

E [a_{i j}] = 0 for all i, j, E [a_{i j}^{2}] = \frac{1}{n} for all i \neq j,

(5)

and

E [{∣ ∣ a_{i j} ∣ ∣}_{i j}^{k}] \leq \frac{C^{k}}{n d^{(k - 2) / 2}} for all i, j and each k \in [2, (log n)^{10 log log n}],

(6)

where $d \equiv d (n)$ is an $n$ -dependent sparsity parameter and $C$ is an absolute positive constant.

Of particular interest are the following special cases:

Bounded case: The entries are bounded in magnitude by $\frac{C}{\sqrt{n}}$ . Then (6) is fulfilled for $d = n$ and all $k$ .
Sub-Gaussian case: The sub-Gaussian norm of each entry satisfies

$∥ a_{i j} ∥_{ψ_{2}} ≜ sup k \geq 1 k^{- 1 / 2} E {[{∣ ∣ a_{i j} ∣ ∣}_{i j}^{k}]}^{1 / k} = O (1 / \sqrt{n}) .$ (7)

It is easily checked that (6) is satisfied for $d = n / (log n)^{11 log log n}$ and all large $n$ .
Erdős-Rényi graphs with edge probability $p \equiv p (n)$ . We may center and scale the adjacency matrix $A$ such that $a_{i j} \sim (B e r n (p) - p) / \sqrt{n p (1 - p)}$ for $i \neq j$ , which satisfies (5) and (6) for $d = n p (1 - p)$ (cf. lmm:AB).

With the moment conditions eq:centerscale and eq:momentcond specified, we are ready to introduce the correlated Wigner model, which encompasses the correlated Erdős-Rényi graph model proposed in

[PG11] as a special case.

Definition 1.1 (Correlated Wigner model).

Let $n$ be a positive integer, $σ \in [0, 1]$ an ( $n$ -dependent) noise parameter, $π_{*}$ a latent permutation on $[n]$ , and $Π_{*} \in {0, 1}^{n \times n}$ the corresponding permutation matrix such that $(Π_{*})_{i π_{*} (i)} = 1$ . Suppose that ${(a_{i j}, b_{π_{*} (i) π_{*} (j)}) : i \leq j}$ are independent pairs of random variables such that both $A = (a_{i j})$ and $B = (b_{i j})$ satisfy (5) and (6),

E [a_{i j} b_{π_{*} (i) π_{*} (j)}] \geq \frac{1 - σ^{2}}{n} for all i \neq j,

(8)

and for a constant $C > 0$ , any $D > 0$ , and all $n \geq n_{0} (D)$ ,

P {∥ ∥ A - Π_{*} B Π_{*}^{⊤} ∥ ∥ \leq C σ} \geq 1 - n^{- D}

(9)

where $∥ \cdot ∥$ denotes the spectral norm.

The parameter $σ$ measures the effective noise level in the model. In the special case of sparse Erdős-Rényi model, $A$ and $B$ are the centered and normalized adjacency matrices of two Erdős-Rényi graphs, which differ by a fraction $2 σ^{2}$ of edges approximately.

In this paper, we prove the following exact recovery guarantee for GRAMPA:

Theorem (Informal statement).

For the correlated Wigner model, if $d \geq polylog (n)$ and $σ \leq c {(log n)}^{- 2 κ}$ for any fixed constant $κ > 2$ and a sufficiently small constant $c > 0$ , then GRAMPA with $η = 1 / polylog n$ recovers $π_{*}$ exactly with high probability for large $n$ . If furthermore $a_{i j}$ and $b_{i j}$ are sub-Gaussian and satisfy (7), then this holds with $κ = 1$ .

This theorem generalizes the exact recovery guarantee for GRAMPA proved in [FMWX19] for the Gaussian Wigner model, albeit at the expense of a slightly stronger requirement for $σ$ than in the Gaussian case. The requirement that $d \geq polylog (n)$ and $σ \leq 1 / polylog (n)$ is the state-of-the-art for polynomial time algorithms on sparse Erdős-Rényi graphs [DMWX18], although we note that the recent work of [BCL $^{+}$ 18] provided an algorithm with super-polynomial runtime $n^{O (log n)}$ that achieves exact recovery when $d \geq n^{o (1)}$ under the much weaker condition of $σ \leq 1 - {(log n)}^{- o (1)}$ .

The analysis in [FMWX19] relies heavily on the rotational invariance of Gaussian Wigner matrices, and does not extend to non-Gaussian models. Here, instead, our universality analysis uses a resolvent representation of the GRAMPA similarity matrix eq:specrep via a contour integral (cf. Proposition 3.2). Capitalizing on local laws for the resolvent of sparse Wigner matrices [EKYY13a, EKYY13b], we show that the similarity matrix eq:specrep is with high probability diagonal dominant in the sense that ${min}_{k} X_{k π_{*} (k)} > {max}_{ℓ \neq π_{*} (k)} X_{k ℓ}$ . This enables rounding procedures as simple as thresholding to succeed.

From an optimization point of view, GRAMPA can also be interpreted as solving a regularized quadratic programming (QP) relaxation of the QAP. More precisely, the QAP eq:QAP can be equivalently written as

min Π \in S_{n} ∥ A Π - Π B ∥_{F}^{2},

(10)

and the similarity matrix $X$ in eq:specrep is a positive scalar multiple of the solution $˜ X$ to

	$argmin X \in R^{n \times n}$	$∥ A X - X B ∥_{F}^{2} + η^{2} ∥ X ∥_{F}^{2}$
	s.t.	$1^{⊤} X 1 = n .$		(11)

(See [FMWX19, Corollary 2.2].) This is a convex relaxation of the program eq:qap-equiv with an additional ridge regularization term. As a result, our analysis immediately yields the same exact recovery guarantees for algorithms that round the solution $˜ X$ to eq:regQP instead of $X$ . In Section 6, we study a tighter relaxation of the QAP eq:qap-equiv that imposes row-sum constraints, and establish the same exact recovery guarantees (up to universal constants) by employing similar technical tools.

Organization

The rest of the paper is organized as follows. In Section 2, we state the main exact recovery guarantees for GRAMPA under the correlated Wigner model, as well as the results specialized to the (sparse) Erdős-Rényi model. We start the analysis by introducing the key resolvent representation of the GRAMPA similarity matrix in Section 3. As a preparation for the main proof, Section 4 provides the needed tools from random matrix theory. The proof of correctness for GRAMPA is then presented in Section 5. In Section 6, we extend the theoretical guarantees to a tighter QP relaxation. Finally, Section 7 is devoted to proving the resolvent bounds which form the main technical ingredient to our proofs.

Notation

Let $[n] ≜ {1, \dots, n}$ . Let $i = \sqrt{- 1}$ . In a Euclidean space $R^{n}$ or $C^{n}$ , let $e_{i}$ be the $i$

-th standard basis vector, and let

1 = 1_{n}

be the all-ones vector. Let

J = J_{n}

denote the

n \times n

all-ones matrix, and let

I = I_{n}

denote the

n \times n

identity matrix. The subscripts are often omitted when there is no ambiguity.

The inner product of $u, v \in C^{n}$ is defined as $⟨ u, v ⟩ = u^{*} v$ . Similarly, for matrices, $⟨ A, B ⟩ = Tr (A^{*} B)$ . Let $∥ v ∥ \equiv ∥ v ∥_{2} = ⟨ v, v ⟩$ and $∥ v ∥_{\infty} = {sup}_{i} | v_{i} |$ for vectors. Let $∥ M ∥ \equiv ∥ M ∥_{op} = {sup}_{v : ∥ v ∥ = 1} ∥ M v ∥$ , $∥ M ∥_{F} = ⟨ M, M ⟩$ , and $∥ M ∥_{\infty} = {sup}_{i, j} | M_{i j} |$ for matrices.

Let $x \land y = min (x, y)$ and $x \lor y = max (x, y)$ . We use $C, C^{'}, c, c^{'}, \dots$ to denote positive constants that may change from line to line. For sequences of positive real numbers $(a_{n})_{n = 1}^{\infty}$ and $(b_{n})_{n = 1}^{\infty}$ , we write $a_{n} ≲ b_{n}$ (resp. $a_{n} ≳ b_{n}$ ) if there is a constant $C > 0$ such that $a_{n} \leq C b_{n}$ (resp. $b_{n} \leq C a_{n}$ ) for all $n \geq 1$ , $a_{n} ≍ b_{n}$ if both relations $a_{n} ≲ b_{n}$ and $a_{n} ≳ b_{n}$ hold, and $a_{n} ≪ b_{n}$ if $a_{n} / b_{n} \to 0$ as $n \to \infty$ . We write $a_{n} = O (b_{n})$ if $| a_{n} | ≲ b_{n}$ and $a_{n} = o (b_{n})$ if $| a_{n} | ≪ b_{n}$ .

2 Exact recovery guarantees for GRAMPA

In this section, we state the the exact recovery guarantees for GRAMPA, making the earlier informal statement precise.

Theorem 2.1.

Fix constants $a > 0$ and $κ > 2$ , and let $η \in [1 / (log n)^{a}, 1]$ . Consider the correlated Wigner model with $n \geq d \geq (log n)^{c_{0}}$ where $c_{0} > max (32 + 4 a, 4 + 7 a)$ . Then there exist $(a, κ)$ -dependent constants $C, n_{0} > 0$ and a deterministic quantity $r (n) \equiv r (n, η, d, a)$ satisfying $r (n) \to 0$ as $n \to \infty$ , such that for all $n \geq n_{0}$ , with probability at least $1 - n^{- 10}$ , the matrix $X$ in eq:specrep satisfies

	$max ℓ \neq π_{*} (k) \| X_{k ℓ} \| \leq C (log n)^{κ} \frac{1}{\sqrt{η}},$
	$max k ∣ ∣ ∣ X_{k π_{*} (k)} - \frac{1 - σ^{2}}{η} ∣ ∣ ∣ \leq C (\frac{r (n)}{η} + \frac{σ}{η^{2}} + (log n)^{κ} \frac{1}{\sqrt{η}}) .$		(12)

If there is a universal constant $K$ for which $a_{i j}$ and $b_{i j}$ are sub-Gaussian with $∥ a_{i j} ∥_{ψ_{2}}, ∥ b_{i j} ∥_{ψ_{2}} \leq K / \sqrt{n}$ , then the above holds also with $κ = 1$ .

As an immediate corollary, we obtain the following exact recovery guarantee for GRAMPA.

Corollary 2.2 (Universal graph matching).

Under the conditions of Theorem 2.1, there exist constants $c, c^{'} > 0$ such that for all $n \geq n_{0}$ , if

(log n)^{- a} \leq η \leq c (log n)^{- 2 κ} and σ \leq c^{'} η,

(13)

then with probability at least $1 - n^{- 10}$ ,

min k X_{k π_{*} (k)} > max ℓ \neq π_{*} (k) X_{k ℓ},

(14)

and hence $ˆ Π$ which solves the linear assignment problem (4) equals $Π_{*}$ .

Proof.

Let $c = 1 / (64 C^{2})$ and $c^{'} = 1 / (2 C)$ , where $C$ is the constant given in thm:diag-dom. Then under assumption eq:assump_sigma_main, we have

C (log n)^{κ} \sqrt{η} \leq C (log n)^{κ} \frac{\sqrt{c}}{(log n)^{κ}} = C \sqrt{c} \leq 1 / 8,

so ${max}_{ℓ \neq π_{*} (k)} | X_{k ℓ} | \leq 1 / (8 η)$ . We also have $C σ / η \leq C c^{'} = 1 / 2$ and $1 - σ^{2} > 7 / 8$ and $C r (n) < 1 / 8$ for all large $n$ , so that ${max}_{k} X_{k π_{*} (k)} > (7 / 8 - 1 / 8 - 1 / 2 - 1 / 8) / η > 1 / (8 η)$ . This implies $\prettyrefeq:wignerseparation$ . ∎

An important application of the above universality result is matching two correlated sparse Erdős-Rényi graphs. Let $G$ be an Erdős-Rényi graph with $n$ vertices and edge probability $q$ , denoted by $G \sim G (n, q)$ . Let $A$ and $B^{'}$ be two copies of Erdős-Rényi graphs that are i.i.d. conditional on $G$ , each of which is obtained from $G$ by deleting every edge of $G$ with probability $1 - s$ independently where $s \in [0, 1]$ . Then we have that $A, B^{'} \sim G (n, p)$ marginally where $p ≜ q s$ . Equivalently, we may first sample an Erdős-Rényi graph $A \sim G (n, p)$ , and then define $B^{'}$ by

B_{i j}^{'} \sim {\begin{matrix} B e r n (s) & if A_{i j} = 1 B e r n (\frac{p (1 - s)}{1 - p}) & if A_{i j} = 0. \end{matrix}

Suppose that we observe a pair of graphs $A$ and $B = Π_{*}^{⊤} B^{'} Π_{*}$ , where $Π_{*}$ is an unknown permutation matrix. We then wish to recover the permutation matrix $Π_{*}$ .

We transform the adjacency matrices $A$ and $B$ so that they satisfy the moment conditions (5) and (6): Define the centered, rescaled versions of $A$ and $B$ by

A ≜ (n p (1 - p))^{- 1 / 2} (A - E [A]) and B ≜ (n p (1 - p))^{- 1 / 2} (B - E [B]) .

(15)

Then (5) clearly holds, and we check the following additional properties.

Lemma 2.3.

For all large $n$ , the matrices $A = (a_{i j})$ and $B = (b_{i j})$ satisfy (6), (8), and eq:diffnorm with $d = n p (1 - p)$ and

σ^{2} = max (\frac{1 - s}{1 - p}, \frac{(log n)^{7}}{d}) .

Proof.

Assume without loss of generality that $Π_{*}$ is the identity matrix. For any $k \geq 2$ we have

E [{∣ ∣ a_{i j} ∣ ∣}_{i j}^{k}] = (n p (1 - p))^{- k / 2} [p {(1 - p)}^{k} + (1 - p) p^{k}] = \frac{{(1 - p)}^{k - 1} + p^{k - 1}}{n d^{(k - 2) / 2}} \leq \frac{1}{n d^{(k - 2) / 2}} .

Thus, the moment condition (6) is satisfied. In addition, we have that for all $i < j$ ,

E [a_{i j} b_{i j}]

= \frac{1}{d} E [(A_{i j} - p) (B_{i j} - p)] = \frac{1}{d} (p s - p^{2}) = \frac{s - p}{n (1 - p)} \leq \frac{1 - σ^{2}}{n},

where the last equality holds by the choice of $σ^{2}$ . Thus, (8) is satisfied. Moreover, let $Δ_{i j} = \frac{1}{\sqrt{2 σ^{2}}} (a_{i j} - b_{i j}) .$ It follows that $E [Δ_{i j}] = 0$ and

E [{∣ ∣ Δ_{i j} ∣ ∣}_{i j}^{k}] = \frac{2 p (1 - s)}{{(2 σ^{2} d)}^{k / 2}} \leq \frac{1}{n (2 σ^{2} d)^{(k - 2) / 2}}

where the last inequality is due to $σ^{2} \geq \frac{1 - s}{1 - p}$ . Thus, by applying lmm:normbound and $2 (log n)^{7} \leq 2 σ^{2} d \leq n$ where the upper bound follows from $p (1 - s) \leq s (1 - s) \leq 1 / 4$ , there exists a constant $C > 0$ such that for any $D > 0$ , with probability at least $1 - n^{- D}$ for all $n \geq n_{0} (D)$ , we have $∥ Δ ∥ \leq C$ and hence $∥ A - B ∥ \leq \sqrt{2} C σ$ . Thus eq:diffnorm is satisfied. ∎

Combining lmm:AB with cor:general immediately yields a sufficient condition for GRAMPA to exactly recover $Π_{*}$ in the correlated Erdős-Rényi graph model.

Corollary 2.4 (Erdős-Rényi graph matching).

Suppose that either

(dense case)

$δ \leq p \leq 1 - δ, \frac{1 - s}{1 - p} \leq (log n)^{- c_{1}}$

for constants $δ \in (0, 1)$ and $c_{1} > 4$ , or
(sparse case)

$n p (1 - p) \geq (log n)^{c_{0}}, \frac{1 - s}{1 - p} \leq (log n)^{- c_{1}}$

for constants $c_{0} > 48$ and $c_{1} > 8$ .

There exist $(δ, c_{0}, c_{1})$ -dependent constants $a, n_{0} > 0$ such that if $η = (log n)^{- a}$ and $n \geq n_{0}$ , then with probability at least $1 - n^{- 10}$ ,

min k X_{k π_{*} (k)} > max ℓ \neq π_{*} (k) X_{k ℓ},

and hence the solution $ˆ Π$ to the linear assignment problem (4) coincides with $Π_{*}$ .

Proof.

For (a), pick $κ = 1$ and any $a$ such that $c_{1} / 2 > a > 2 κ = 2$ . For (b), pick any $a, κ$ such that $c_{1} / 2 > a > 2 κ > 4$ and $c_{0} > 32 + 4 a > 4 + 7 a$ . Then all conditions of thm:diag-dom and cor:general are satisfied for large $n$ , and the result follows. ∎

3 Resolvent representation

For a real symmetric matrix $A$ with spectral decomposition (2), its resolvent is defined by

R_{A} (z) ≜ (A - z I)^{- 1} = \sum i \frac{1}{λ_{i} - z} v_{i} v_{i}^{⊤}

for $z \in C ∖ R$ . Then we have the matrix symmetry $R_{A} (z)^{⊤} = R_{A} (z)$ , conjugate symmetry $¯ ¯¯¯¯¯¯¯¯¯¯¯¯ ¯ R_{A} (z) = R_{A} (¯ z)$ , and the following Ward identity.

Lemma 3.1 (Ward identity).

For any $z \in C ∖ R$ and any real symmetric matrix $A$ ,

R_{A} (z) ¯ ¯¯¯¯¯¯¯¯¯¯¯¯ ¯ R_{A} (z) = \frac{Im R_{A} (z)}{Im z} .

Proof.

By the definition of $R (z) \equiv R_{A} (z)$ and conjugate symmetry, it holds

\frac{Im R (z)}{Im z} = \frac{R (z) - ¯ ¯¯¯¯¯¯¯¯¯ ¯ R (z)}{z - ¯ z} = \frac{(A - z I)^{- 1} - (A - ¯ z I)^{- 1}}{z - ¯ z} = (A - z I)^{- 1} (A - ¯ z I)^{- 1} = R (z) ¯ ¯¯¯¯¯¯¯¯¯ ¯ R (z) .

∎

The following resolvent representation of $X$ is central to our analysis.

Proposition 3.2.

Consider symmetric matrices $A$ and $B$ with spectral decompositions (2), and suppose that $∥ A ∥ \leq 2.5$ . Then the matrix $X$ defined in eq:specrep admits the following representation

X = \frac{1}{2 π} Re \oint_{Γ} R_{A} (z) J R_{B} (z + i η) d z,

(16)

where

Γ = {z : | Re z | = 3 and | Im z | \leq η / 2 or | Im z | = η / 2 and | Re z | \leq 3}

(17)

is the rectangular contour with vertices $\pm 3 \pm i η / 2$ .

Proof.

We have

$X$	$= η \sum i, j v_{i} v_{i}^{⊤} J \frac{w_{j} w_{j}^{⊤}}{(λ_{i} - μ_{j})^{2} + η^{2}}$

	$= Im \sum i v_{i} v_{i}^{⊤} J R_{B} (λ_{i} + i η)$	(18)

by Lemma 3.1. Consider the function $f : C \to C^{n \times n}$ defined by $f (z) = J R_{B} (z + i η)$ . Then each entry $f_{k ℓ}$ is analytic in the region ${z : Im z > - η}$ . Since $Γ$

encloses each eigenvalue

λ_{i}

A

, the Cauchy integral formula yields entrywise equality

- \frac{1}{2 π i} \oint_{Γ} \frac{f (z)}{λ_{i} - z} d z = f (λ_{i}) .

(19)

Substituting this into (18), we obtain

X = Im \sum i v_{i} v_{i}^{⊤} (- \frac{1}{2 π i} \oint_{Γ} \frac{f (z)}{λ_{i} - z} d z) = \frac{1}{2 π} Re \oint_{Γ} R_{A} (z) f (z) d z,

(20)

which completes the proof in view of the definition of $f$ . ∎

4 Tools from random matrix theory

Before proving our main results, we introduce the relevant tools from random matrix theory. In particular, the resolvent bounds in Theorem 4.5 constitute an important technical ingredient in our analysis.

4.1 Concentration inequalities

We start with some known concentration inequalities in the literature.

Lemma 4.1 (Norm bounds).

For any constant $ε > 0$ and a universal constant $c > 0$ , if $n \geq d \geq (log n)^{6 + 6 ε}$ , then with probability at least $1 - e^{- c (log n)^{1 + ε}}$ ,

∥ A ∥ \leq 2 + \frac{(log n)^{1 + ε}}{d^{1 / 4}} .

Proof.

See [EKYY13b, Lemma 4.3], where we fix the parameter $ξ = 1 + ε$ in [EKYY13b, Eq. (2.4)]. The notational identification is $q \equiv \sqrt{d}$ . ∎

Lemma 4.2 (Concentration inequalities).

Let $α, β \in R^{n}$ be independent random vectors with independent entries, satisfying

E [α_{i}] = E [β_{i}] = 0, E [α_{i}^{2}] = E [β_{i}^{2}] = \frac{1}{n},

max (E [| α_{i} |^{k}], E [| β_{i} |^{k}]) \leq \frac{1}{n d^{(k - 2) / 2}} for each k \in [2, (log n)^{10 log log n}] .

(21)

For any constant $ε > 0$ and universal constants $C, c > 0$ , if $n \geq d \geq (log n)^{6 + 6 ε}$ , then:

For each $i \in [n]$ , with probability at least $1 - e^{- c (log n)^{1 + ε}}$ ,

$| α_{i} | \leq \frac{C}{\sqrt{d}} .$ (22)
For any deterministic vector $v \in C^{n}$ , with probability at least $1 - e^{- c (log n)^{1 + ε}}$ ,

$∣ ∣ v^{⊤} α ∣ ∣ \leq (log n)^{1 + ε} (\frac{∥ v ∥_{\infty}}{\sqrt{d}} + \frac{∥ v ∥_{2}}{\sqrt{n}}) .$ (23)

Furthermore, for any even integer $p \in [2, (log n)^{10 log log n}]$ ,

$E [{∣ ∣ v^{⊤} α ∣ ∣}^{p}] \leq (C p)^{p} {(\frac{∥ v ∥_{\infty}}{\sqrt{d}} + \frac{∥ v ∥_{2}}{\sqrt{n}})}^{p} .$ (24)
For any deterministic matrix $M \in C^{n \times n}$ , with probability at least $1 - e^{- c (log n)^{1 + ε}}$ ,

$∣ ∣ ∣ α^{⊤} M α - \frac{1}{n} Tr M ∣ ∣ ∣ \leq (log n)^{2 + 2 ε} (\frac{2 ∥ M ∥_{\infty}}{\sqrt{d}} + \frac{∥ M ∥_{F}}{n})$ (25)

and

$∣ ∣ α^{⊤} M β ∣ ∣ \leq (log n)^{2 + 2 ε} (\frac{2 ∥ M ∥_{\infty}}{\sqrt{d}} + \frac{∥ M ∥_{F}}{n}) .$ (26)

Proof.

See [EKYY13b, Lemma 3.7, Lemma 3.8, and Lemma A.1(i)], where again we fix $ξ = 1 + ε$ . ∎

Next, based on the above lemma, we state concentration inequalities for a bilinear form that apply to our setting directly.

Lemma 4.3 (Concentration of bilinear form).

Let $α, β \in R^{n}$ be random vectors such that the pairs $(α_{i}, β_{i})$ for $i \in [n]$ are independent, with

E [α_{i}] = E [β_{i}] = 0, E [α_{i}^{2}] = E [β_{i}^{2}] = \frac{1}{n}, E [α_{i} β_{i}] \geq \frac{1 - σ^{2}}{n} .

Let $M \in C^{n \times n}$ be any deterministic matrix.

For any constant $ε > 0$ , suppose (21) holds where $n \geq d \geq (log n)^{6 + 6 ε}$ . Then there are universal constants $C, c > 0$ such that with probability at least $1 - e^{- c (log n)^{1 + ε}}$ ,

$∣ ∣ ∣ α^{⊤} M β - \frac{1 - σ^{2}}{n} Tr M ∣ ∣ ∣ \leq C {(log n)}^{2 + 2 ε} (\frac{1}{n} ∥ M ∥_{F} + \frac{1}{\sqrt{d}} ∥ M ∥_{\infty}) .$ (27)
Suppose that $α_{i}, β_{i}$ are sub-Gaussian with $∥ α_{i} ∥_{ψ_{2}} = ∥ β_{i} ∥_{ψ_{2}} \leq \frac{K}{\sqrt{n}}$ for a constant $K > 0$ . Then for any $D > 0$ , there exists a constant $C \equiv C_{K, D}$ only depending on $K$ and $D$ such that with probability at least $1 - n^{- D}$ ,

$∣ ∣ ∣ α^{⊤} M β - \frac{1 - σ^{2}}{n} Tr M ∣ ∣ ∣ \leq \frac{C log n}{n} ∥ M ∥_{F} .$ (28)

Proof.

In view of the polarization identity

α^{⊤} M β = \frac{1}{4} (α + β)^{⊤} M (α + β) - \frac{1}{4} (α - β)^{⊤} M (α - β),

it suffices to analyze the two terms separately. Note that

E [(α + β)^{⊤} M (α + β)] = \frac{4 - 2 σ^{2}}{n} Tr M, E [(α - β)^{⊤} M (α - β)] = \frac{2 σ^{2}}{n} Tr M,

which yields the desired expectation $E [α^{⊤} M β] = \frac{1 - σ^{2}}{n} Tr M .$ Thus it remains to study the deviation.

To prove the concentration bound eq:diag, we obtain from eq:quad1 that, there is a universal constant $c > 0$ such that with probability at least $1 - e^{- c (log n)^{1 + ε}}$ ,

∣ ∣ (α \pm β)^{⊤} M (α \pm β) - E [(α \pm β)^{⊤} M (α \pm β)] ∣ ∣ \leq (log n)^{2 + 2 ε} (\frac{1}{n} ∥ M ∥_{F} + \frac{2}{\sqrt{d}} ∥ M ∥_{\infty}),

from which eq:diag easily follows.

The sub-Gaussian concentration bound eq:bilinear_wigner follows from the Hanson-Wright inequality [HW71, RV13]. More precisely, note that $max {∥ α + β ∥_{ψ_{2}}, ∥ α - β ∥_{ψ_{2}}} \leq ∥ α ∥_{ψ_{2}} + ∥ β ∥_{ψ_{2}} \leq 2 K / \sqrt{d}$ , so taking $δ = n^{- D} / 2$ in [FMWX19, Lemma A.2] yields that with probability at least $1 - n^{- D}$ ,

∣ ∣ (α \pm β)^{⊤} M (α \pm β) - E [(α \pm β)^{⊤} M (α \pm β)] ∣ ∣ \leq C_{K, D} \frac{log n}{} n ∥ M ∥_{F},

which completes the proof. ∎

4.2 The Stieltjes transform

Denote the semicircle density and its Stieltjes transform by

ρ (x) = \frac{1}{2 π} \sqrt{4 - x^{2}} 1_{{| x | \leq 2}} and m_{0} (z) = \int \frac{1}{x - z} ρ (x) d x = \frac{- z + \sqrt{z^{2} - 4}}{2}

(29)

respectively, where $m_{0} (z)$ is defined for $z \notin [- 2, 2]$ , and $\sqrt{z^{2} - 4}$ is defined with a branch cut on $[- 2, 2]$ so that $\sqrt{z^{2} - 4} \sim z$ as $| z | \to \infty$ . We have the conjugate symmetry $¯ ¯¯¯¯¯¯¯¯¯¯¯¯ ¯ m_{0} (z) = m_{0} (¯ z)$ .

We record the following basic facts about the Stieltjes transform.

Proposition 4.4.

For each $z \in C ∖ R$ , the Stieltjes transform $m_{0} (z)$ is the unique value satisfying

m_{0} (z)^{2} + z m_{0} (z) + 1 = 0 and Im m_{0} (z) \cdot Im z > 0.

(30)

Setting $ζ (z) ≜ min (| Re z - 2 |, | Re z + 2 |)$ , uniformly over $z \in C ∖ [- 2, 2]$ with $| z | \leq 10$ ,

| m_{0} (z) | ≍ 1, | Im m_{0} (z) | ≳ | Im z |, and | Im m_{0} (z) | ≍ {\begin{matrix} \sqrt{ζ (z) + | Im z |} & if | Re z | \leq 2, | Im z | / \sqrt{ζ (z) + | Im z |} & if | Re z | > 2. \end{matrix}

(31)

For $x \in [- 2, 2]$ , the continuous extensions

m_{0}^{+} (x) ≜ lim z \to x : z \in C^{+} m_{0} (z), m_{0}^{-} (x) ≜ lim z \to x : z \in C^{-} m_{0} (z)

from $C^{+}$ and $C^{-}$ both exist. For all $x \in [- 2, 2]$ , these satisfy

m_{0}^{\pm} (x)^{2} + x m_{0}^{\pm} (x) + 1 = 0, m_{0}^{+} (x) = ¯ ¯¯¯¯¯¯¯¯¯¯¯¯¯ ¯ m_{0}^{-} (x), \frac{1}{π} Im m_{0}^{+} (x) = - \frac{1}{π} I m m_{0}^{-} (x) = ρ (x), | m_{0}^{\pm} (x) | = 1.

(32)

Proof.

(30) follows from the definition of $m_{0}$ . (31) follows from [EKYY13a, Lemma 4.3] and continuity and conjugate symmetry of $m_{0}$ . For the existence of $m_{0}^{+}$ (and hence also $m_{0}^{-}$ ), see e.g. the more general statement of [Bia97, Corollary 1]. The first claim of (32) follows from continuity and (30), the second from conjugate symmetry, the third from the Stieltjes inversion formula, and the last from the fact that the two roots of (30) at $z = x \in [- 2, 2]$ are $m_{0}^{+} (x)$ and $m_{0}^{-} (x) = ¯ ¯¯¯¯¯¯¯¯¯¯¯¯¯ ¯ m_{0}^{+} (x)$ , so that $1 = m_{0}^{\pm} (x) ¯ ¯¯¯¯¯¯¯¯¯¯¯¯¯ ¯ m_{0}^{\pm} (x) = | m_{0}^{\pm} (x) |^{2}$ . ∎

4.3 Resolvent bounds

For a fixed constant $a > 0$ and all large $n$ , we bound the resolvent $R (z) = R_{A} (z)$ over the spectral domain

	$D$	$= D_{1} \cup D_{2}, where$
	$D_{1}$	$= {z \in C : Re z \in [- 3, 3], \| I m z \| \in [1 / (log n)^{a}, 1]}, and$
	$D_{2}$	$= {z \in C : \| Re z \| \in [2.6, 3], \| I m z \| \leq 1 / (log n)^{a}} .$

Here, $D_{1}$ is the union of two strips in the upper and lower half planes, and $D_{2}$ is the union of two strips in the left and right half planes.

Theorem 4.5 (Resolvent bounds).

Suppose $A \in R^{n \times n}$ has independent entries $(a_{i j})_{i \leq j}$ satisfying (5) and (6). Fix a constant $a > 0$ which defines the domain $D$ , fix $ε > 0$ , and set

b = max (16 + 3 ε + 2 a, 3 + 3 ε + 5 a / 2), b^{'} = max (16 + 4 ε + 2 a, 4 + 5 ε + 6 a) .

Suppose $n \geq d \geq (log n)^{b^{'}}$ . Then for some constants $C, c, n_{0} > 0$ depending on $a$ and $ε$ , and for all $n \geq n_{0}$ , with probability $1 - e^{- c (log n) (log log n)}$ , the following hold simultaneously for every $z \in D$ :

(Entrywise bound) For all $j \neq k \in [n]$ ,

$| R_{j k} (z) | \leq \frac{C (log n)^{2 + 2 ε + a}}{\sqrt{d}} .$ (33)

For all $j \in [n]$ ,

$| R_{j j} (z) - m_{0} (z) | \leq \frac{C (log n)^{2 + 2 ε + 3 a / 2}}{\sqrt{d}} .$ (34)
(Row sum bound) For all $j \in [n]$ ,

$∣ ∣ e_{j}^{⊤} R (z) 1 ∣ ∣ \leq C (log n)^{1 + ε + a} .$ (35)
(Total sum bound)

$| 1^{⊤} R (z) 1 - n \cdot m_{0} (z) | \leq \frac{C n (log n)^{b}}{\sqrt{d}} .$ (36)

The proof follows ideas of [EKYY13b], and we defer this to Section 7. As the spectral parameter $z$ is allowed to converge to the interval $[- 2, 2]$ with increasing $n$ , this type of result is often called a “local law” in the random matrix theory literature. The focus of the above is a bit different from the results stated in [EKYY13b], as we wish to obtain explicit logarithmic bounds for $| Im z | ≍ 1$

Spectral Graph Matching and Regularized Quadratic Relaxations II: Erdős-Rényi Graphs and Universality

Authors

Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model

Seeded graph matching for the correlated Wigner model via the projected power method

Testing network correlation efficiently via counting trees

Graph matching: relax or not?

Tractable Graph Matching via Soft Seeding

Correlated Stochastic Block Models: Exact Graph Matching with Applications to Recovering Communities

Seedless Graph Matching via Tail of Degree Distribution for Correlated Erdos-Renyi Graphs

1 Introduction

Definition 1.1 (Correlated Wigner model).

Theorem (Informal statement).

Organization

Notation

2 Exact recovery guarantees for GRAMPA

Theorem 2.1.

Corollary 2.2 (Universal graph matching).

Proof.

Lemma 2.3.

Proof.

Corollary 2.4 (Erdős-Rényi graph matching).

Proof.

3 Resolvent representation

Lemma 3.1 (Ward identity).

Proof.

Proposition 3.2.

Proof.

4 Tools from random matrix theory

4.1 Concentration inequalities

Lemma 4.1 (Norm bounds).

Proof.

Lemma 4.2 (Concentration inequalities).

Proof.

Lemma 4.3 (Concentration of bilinear form).

Proof.

4.2 The Stieltjes transform

Proposition 4.4.

Proof.

4.3 Resolvent bounds

Theorem 4.5 (Resolvent bounds).

Spectral Graph Matching and Regularized Quadratic Relaxations II: Erdős-Rényi Graphs and Universality

Authors

Related Research

Spectral Graph Matching and Regularized Quadratic Relaxations I: The Gaussian Model

Seeded graph matching for the correlated Wigner model via the projected power method

Testing network correlation efficiently via counting trees

Graph matching: relax or not?

Tractable Graph Matching via Soft Seeding

Correlated Stochastic Block Models: Exact Graph Matching with Applications to Recovering Communities

Seedless Graph Matching via Tail of Degree Distribution for Correlated Erdos-Renyi Graphs

This week in AI

1 Introduction

Definition 1.1 (Correlated Wigner model).

Theorem (Informal statement).

Organization

Notation

2 Exact recovery guarantees for GRAMPA

Theorem 2.1.

Corollary 2.2 (Universal graph matching).

Proof.

Lemma 2.3.

Proof.

Corollary 2.4 (Erdős-Rényi graph matching).

Proof.

3 Resolvent representation

Lemma 3.1 (Ward identity).

Proof.

Proposition 3.2.

Proof.

4 Tools from random matrix theory

4.1 Concentration inequalities

Lemma 4.1 (Norm bounds).

Proof.

Lemma 4.2 (Concentration inequalities).

Proof.

Lemma 4.3 (Concentration of bilinear form).

Proof.

4.2 The Stieltjes transform

Proposition 4.4.

Proof.

4.3 Resolvent bounds

Theorem 4.5 (Resolvent bounds).