Hermitian Laplacians and a Cheeger inequality for the Max-2-Lin problem

11/27/2018 ∙ by Huan Li, et al. ∙ 0

We study spectral approaches for the MAX-2-LIN(k) problem, in which we are given a system of m linear equations of the form x_i - x_j ≡ c_ij k, and required to find an assignment to the n variables {x_i} that maximises the total number of satisfied equations. We consider Hermitian Laplacians related to this problem, and prove a Cheeger inequality that relates the smallest eigenvalue of a Hermitian Laplacian to the maximum number of satisfied equations of a MAX-2-LIN(k) instance I. We develop an O(kn^2) time algorithm that, for any (1-ε)-satisfiable instance, produces an assignment satisfying a (1 - O(k)√(ε))-fraction of equations. We also present a subquadratic-time algorithm that, when the graph associated with I is an expander, produces an assignment satisfying a (1- O(k^2)ε)-fraction of the equations. Our Cheeger inequality and first algorithm can be seen as generalisations of the Cheeger inequality and algorithm for MAX-CUT developed by Trevisan.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

In the MAX-2-LIN() problem, we are given a system of linear equations of the form

(1.1)

where and each equation has weight . The objective is to find an assignment to the variables that maximises the total weight of satisfied equations. As an important case of Unique Games [FL92, Kho02], the MAX-2-LIN() problem has been extensively studied in theoretical computer science. This problem is known to be NP-hard to approximate within a ratio of for any constant  [FR04, Hås01], and it is conjectured to be hard to distinguish between MAX-2-LIN() instances for which a -fraction of equations can be satisfied versus instances for which only an -fraction can be satisfied [KKMO07]. On the algorithmic side, there has been a number of LP and SDP-based algorithms proposed for the MAX-2-LIN() problem (e.g., [Kho02, Tre05, CMM06, GT06]), and the case of , which corresponds to the classical MAX-CUT problem for undirected graphs [GW95, Kar72], has been widely studied over the past fifty years.

In this paper we investigate efficient spectral algorithms for MAX-2-LIN(). For any MAX-2-LIN() instance with variables, we express by a Hermitian Laplacian matrix , and analyse the spectral properties of . In comparison to the well-known Laplacian matrix for undirected graphs [Chu97], complex-valued entries in are able to express directed edges in the graph associated with , and at the same time ensure that all the eigenvalues of are real-valued. We demonstrate the power of our Hermitian Laplacian matrices by relating the maximum number of satisfied equations of to the spectral properties of . In particular, we develop a Cheeger inequality that relates partial assignments of to , the smallest eigenvalue of . Based on a recursive application of the algorithm behind our Cheeger inequality, as well as a spectral sparsification procedure for MAX-2-LIN() instances, we present an approximation algorithm for MAX-2-LIN() that runs in time. Our algorithm is easy to implement, and is significantly faster than most SDP-based algorithms for this problem in the literature, while achieving similar guarantees for constant values of . The formal statement of our result is as follows:

Theorem 1.1.

There is an -time algorithm such that, for any given MAX-2-LIN() instance with optimum , the algorithm returns an assignment satisfying at least a -fraction of the equations555 An instance has optimum , if the maximum fraction of the total weights of satisfied equations is ..

Our result can be viewed as a generalisation of the MAX-CUT algorithm by Trevisan [Tre12], who derived a Cheeger inequality that relates the value of the maximum cut to the smallest eigenvalue of an undirected graph’s adjacency matrix. The proof of Trevisan’s Cheeger inequality, however, is based on constructing sweep sets in , while in our setting constructing sweep sets in is needed, as the underlying graph defined by

is directed and eigenvectors of

are in . The other difference between our result and the one in [Tre12] is that the goal of the MAX-CUT problem is to find a bipartition of the vertex set, while for the MAX-2-LIN() problem we need to use an eigenvector to find vertex-disjoint subsets, which corresponds to subsets of variables assigned to the same value.

Our approach also shares some similarities with the one by Goemans and Williamson [GW04], who presented a -approximation algorithm for MAX-2-LIN(3) based on Complex Semidefinite Programming. The objective function of their SDP relaxation is, in fact, exactly the quadratic form of our Hermitian Laplacian matrix , although this matrix was not explicitly defined in their paper. In addition, their rounding scheme divides the complex unit ball into

regions according to the angle with a random vector, which is part of our rounding scheme as well. Therefore, if one views Trevisan’s work 

[Tre12] as a spectral analogue to the celebrated SDP-based algorithm for MAX-CUT by Goemans and Williamson [GW95], our result can be seen as a spectral analogue to the Goemans and Williamson’s algorithm for MAX-2-LIN().

We further prove that, when the undirected graph associated with a MAX-2-LIN() instance is an expander, the approximation ratio from Theorem 1.1 can be improved. Our result is formally stated as follows:

Theorem 1.2.

Let be an instance of MAX-2-LIN() on a -regular graph with vertices and suppose its optimum is . There is an -time algorithm that returns an assignment satisfying at least a

(1.2)

fraction of equations in , where is the second smallest eigenvalue of the normalised Laplacian matrix of the underlying undirected graph .

Our technique is similar to the one by Kolla [Kol11], which was used to show that solving the MAX-2-LIN() problem on expander graphs is easier. In [Kol11], a MAX-2-LIN() instance is represented by the label-extended graph, and the algorithm is based on an exhaustive search in a subspace spanned by eigenvectors associated with eigenvalues close to . When the underlying graph of the MAX-2-LIN() instance has good expansion, this subspace is of dimension . Therefore, the exhaustive search runs in time , which is polynomial-time when . Comparing with the work in [Kol11], we show that, when the underlying graph has good expansion, the eigenvector associated with the smallest eigenvalue of the Hermitian Laplacians suffices to give a good approximation. We notice that Arora et al. [AKK08] already showed that, for expander graphs, it is possible to satisfy a fraction of equations in polynomial time without any dependency on . Their algorithm is based on an SDP relaxation.

Other related work.

There are many research results for the MAX-2-LIN() problem (e.g., [Kho02, Tre05, CMM06, GT06]), and we briefly discuss the ones most closely related to our work. For the MAX-2-LIN() problem and Unique Games, spectral techniques are usually employed to analyse the Laplacian matrix of the so-called Label-Extended graphs. Apart from the above-mentioned result [Kol11], Arora, Barak and Steurer [ABS15] obtained an -time algorithm for Unique Games, whose algorithm makes use of Label-Extended graphs as well. We also notice that the adjacency matrix corresponding to our Hermitian Laplacian was considered by Singer [Sin11] in relation to an angular synchronisation problem. The connection between the eigenvectors of such matrix and the MAX-2-LIN() problem was also mentioned, but without offering formal approximation guarantees.

2 Hermitian Matrices for Max-2-Lin()

We can write an instance of MAX-2-LIN() by , where denotes a directed graph with an edge weight function and an edge color function , where . More precisely, every equation with weight corresponds to a directed edge with weight and color . In the rest of this paper, we will assume that is weakly connected, and write if there is a directed edge from to . The conjugate transpose of any vector is denoted by .

We define the Hermitian adjacency matrix for instance by

(2.1)

where is the complex -th root of unity, and is its conjugate. We define the degree-diagonal matrix by where is the weighted degree given by

(2.2)

The Hermitian Laplacian matrix is then defined by , and the corresponding normalised Laplacian matrix by . The eigenvalues of any matrix are expressed by . The quadratic forms of can be related to the corresponding instance of MAX-2-LIN() by the following lemma.

Lemma 2.1.

For any vector , we have

(2.3)

and

(2.4)
Proof.

For any vector , we can write

(2.5)

We can also write

(2.6)

Combining these with finishes the proof. ∎

The lemma below presents a qualitative relationship between the eigenvector associated with and an assignment of .

Lemma 2.2.

All eigenvalues of are in the range . Moreover, if and only if there exists an assignment satisfying all equations in .

Proof.

To bound the eigenvalues of , we look at the following Rayleigh quotient

where . By Lemma 2.1, the numerator satisfies

and also

Therefore, the eigenvalues of lie in the range . Moreover, if and only if there exists an such that , i.e.,

holds for all . The existence of such an is equivalent to the existence of an assignment satisfying all equations in . ∎

3 A Cheeger inequality for and Max-2-Lin()

The discrete Cheeger inequality [Alo86] shows that, for any undirected graph , the conductance of can be approximated by the second smallest eigenvalue of ’s normalised Laplacian matrix , i.e.,

(3.1)

Moreover, the proof of the second inequality above is constructive, and indicates that a subset with conductance at most can be found by using the second bottom eigenvector of to embed vertices on the real line. As one of the most fundamental results in spectral graph theory, the Cheeger inequality has found applications in the study of a wide range of optimisation problems, e.g., graph partitioning [LGT14], max-cut [Tre12], and many practical problems like image segmentation [SM00] and web search [Kle99].

In this section, we develop connections between and MAX-2-LIN() by proving a Cheeger-type inequality. Let

be an arbitrary partial assignment of an instance , where means that the assignment of has not been decided. These variables’ assignments will be determined through some recursive construction, which will be elaborated in Section 5. We remark that this framework of recursively computing a partial assignment was first introduced by Trevisan [Tre12], and our theorem can be viewed as a generalisation of the one in [Tre12], which corresponds to the case of ours.

To relate quadratic forms of with the objective function of the MAX-2-LIN() problem, we introduce a penalty function as follows:

Definition 3.1.

Given a partial assignment and a directed edge , the penalty of with respect to is defined by

(3.2)

For simplicity, we write when the underlying instance is clear from the context.

The values of from Definition 3.1 are chosen according to the following facts: (1) If both and ’s values are assigned, then their penalty is if the equation associated with is unsatisfied, and otherwise; (2) If both and ’s values are , then their penalty is temporally set to . Their penalty will be computed when and ’s assignment are determined during a later recursive stage; (3) If exactly one of is assigned, is set to , since a random assignment to the other variable makes the edge

satisfied with probability

.

Without loss of generality, we only consider for which for at least one vertex , and define the penalty of assignment by

(3.3)

where . Notice that the ’s value is multiplied by in accordance with the objective of MAX-2-LIN() which is to maximise the total weight of satisfied assignments. Also, we multiply by in the numerator since edges with at least one assigned endpoint are counted at most twice in . Notice that, as long as is weakly connected, if and only if all edges are satisfied by and, in general, the smaller the value of , the more edges are satisfied by . With this in mind, we define the imperfectness of to quantify how close is to an instance where all equations can be satisfied by a single assignment.

Definition 3.2.

Given any MAX-2-LIN() instance , the imperfectness of is defined by

(3.4)

The main result of this section is a Cheeger-type inequality that relates and , which is summarised in Theorem 3.3. Note that, since for , the factor before in the theorem statement is at most for .

Theorem 3.3.

Let be the smallest eigenvalue of . It holds that

(3.5)

Moreover, given the eigenvector associated with , there is an -time algorithm that returns a partial assignment such that

(3.6)

Our analysis is based on the following fact about the relations about the angle between two vectors and their Euclidean distance. For some , we write to denote the angle from to , i.e., is the unique real number in such that

Fact 3.4.

Let be complex numbers such that . The following statements hold:

  1. If , then it holds that

    (3.7)
  2. If , then it holds that

    (3.8)
Proof.

We assume and prove the first statement. Let

Then we have that

where the last inequality follows by the fact that for any and it holds that . Multiplying on the both sides of the inequality above gives us (3.7).

Now we prove the second statement. We have

where the last inequality follows from the fact that . Dividing both sides of the inequality above by gives us (3.8). ∎

Proof of Theorem 3.3.

We first prove . For a partial assignment , we construct a vector by

(3.9)

Then, we have

(3.10)

where the second line follows from the fact that

(3.11)

always holds for all , and the third line follows from Lemma 2.1 and that

(3.12)

This proves that .

Secondly, we assume that is the vector such that

and prove the existence of an assignment based on satisfying

which will imply (3.5) and (3.6). We scale each coordinate of and without loss of generality assume that . For real numbers and , we define disjoint sets of vertices indexed by as follows:

(3.13)

We then define an assignment where

(3.14)

By definition, the vertex sets correspond to the vectors in the regions of the unit ball after each vector is rotated by radians counterclockwise. The role of is to only consider the coordinates with . This is illustrated in Figure 1.

Set

Set

Set

Random rotation

by

Figure 1: Illustration of the proof for Theorem 3.3 for the case of . The gray circle is obtained by sweeping , and the red arrow represents a random angle . A partial assignment is determined by the values of and .

Our goal is to construct probability distributions for

and such that

(3.15)

This implies by linearity of expectation that

(3.16)

and existence of an assignment satisfying (3.6).

Now let us assume that is chosen such that

follows from a uniform distribution over

, and is chosen uniformly at random from . We analyse the numerator and denominator in the left-hand side of (3.15). For the denominator, it holds that

(3.17)

For the numerator, it holds by linearity of expectation that

(3.18)

Then we look at for every edge . The analysis is based on the value of , the angle from rotated by radians clockwise to .

  • Case 1: . It holds that

    where the second equality follows from that

    the third inequality follows by Fact 3.4 and that equals exactly the angle between and .

  • Case 2: . It holds that

    where the second equality follows from the fact that edge can not be satisfied when is in this range, the first inequality follows by Fact 3.4 and that the angle between and is at least , and the last line follows by the triangle inequality.

Combining these two cases gives us that

(3.19)

where the second inequality follows by the Cauchy-Schwarz inequality. Combining this with (3) finishes the proof of the inequality (3.5).

Finally, let us look at the time needed to find the desired partial assignment. Notice that, by the law of total expectation, we can write