Approximation algorithms and fast (sub)exponential time exact algorithms are among the two most popular approaches employed to tackle NP-hard problems. While both have had their fair share of successes, they seem to hit roadblocks for a number of reasons; the PCP theorem [AS98, ALM98] and the theory of hardness of approximation developed from it have established, for many optimization problems, that trivial algorithms are the best one could hope for (in polynomial time). On the other hand, the Exponential Time Hypothesis (ETH) [IP01, IPZ01] and the fine-grained reductions surrounding it have demonstrated that “brute force” algorithms are, or at least close to, the fastest possible for numerous natural problems.
These barriers have led to studies in the cross-fertilization between the two fields, in which one attempts to apply both techniques simultaneously to overcome known lower bounds. Generally speaking, these works study the trade-offs between the running time of the algorithms and the approximation ratio. In other words, a typical question arising here is: what is the best running time for an algorithm with a given approximation ratio ?
Optimization problems often admit natural “limited brute force” approximation algorithms that use brute force to find the optimal solution restricted to a subset of variables and then extend this to a whole solution. Similar to the study of fast exact algorithms for which a general motivating question is whether one can gain a noticeable speedup over “brute force”, the analogous question when dealing with approximation algorithms is whether one can do significantly better than these limited brute force algorithms.
For example, let us consider the E3SAT problem, which is to determine whether a given 3CNF formula is satisfiable. The brute force (exact) algorithm runs in time, while ETH asserts that it requires time to solve the problem. The optimization version of E3SAT is the Max E3SAT problem, where the goal is to find an assignment that satisfies as many clauses as possible. On the purely approximation front, a trivial algorithm that assigns every variable uniformly independently at random gives 7/8-approximation for Max E3SAT, while Hastad’s seminal work [Hås01] established NP-hardness for obtaining -approximation for any constant . The “limited brute force” algorithm for Max E3SAT chooses a subset of variables, enumerates all possible assignments to those variables and picks values of the remaining variables randomly; this achieves -approximation in time . Interestingly, it is known that running time of is necessary to gain a -approximation if one uses Sum-of-Squares relaxations [Gri01, Sch08, KMOW17], which gives some evidence that the running time of “limited brute force” approximation algorithms for Max E3SAT are close to best possible.
In contrast to Max E3SAT, one can do much better than “limited brute force” for Unique Games. Specifically, Arora et al. [AIMS10] show that one can satisfy an fraction of clauses in a -satisfiable instance of Unique Games in time , a significant improvement over the trivial time “limited brute force” algorithm. This algorithm was later improved by the celebrated algorithm of Arora, Barak and Steurer [ABS15] that runs in time .
A number of approximation problems, such as -approximation of Vertex Cover [KR08, BK09], approximation of Max Cut [KKMO07], and constant approximation of Non-uniform Sparsest Cut [CKK06, KV15] are known to be at least as hard as Unique Games, but are not known to be equivalent to Unique Games. If they were equivalent, the subexponential algorithm of [ABS15] would also extend to these other problems. It is then natural to ask whether these problems admit subexponential time algorithms, or at least “better than brute force” algorithms. Indeed, attempts have been made to design such algorithms [ABS15, GS11], although these algorithms only achieve significant speed-up for specific classes of instances, not all worst case instances.
Recently, Bansal et al. [BCL17] presented a “better than brute force” algorithm for Vertex Cover, which achieve a -approximation in time . Note that the trade-off between approximation and running time is more analogous to the [AIMS10] algorithm for Unique Games than with the “limited brute force” algorithm for Max 3ESAT discussed above.
The algorithm of Bansal et al. is partially combinatorial and is based on a reduction to the Vertex Cover problem in bounded-degree graphs, for which better approximation algorithms are known compared to general graphs. Curiously, the work of Bansal et al. does not subsume the best known polynomial time algorithm for Vertex Cover: Karakostas [Kar09] shows that there is a polynomial time algorithm for Vertex Cover that achieves a approximation ratio, but if one set in the algorithm of Bansal et al. one does not get a polynomial running time.
This overview raises a number of interesting questions: is it possible to replicate, or improve, the vertex cover approximation of Bansal et al. [BCL17] using Sum-of-Square relaxations? A positive result would show that, in a precise sense, approximation of Max 3SAT is “harder” than approximation for Vertex Cover (since the former requires rounds while the latter would be achievable with rounds). Is it possible to have a “better than brute force” approximation algorithm for Vertex Cover that recovers Karakostas’s algorithm as a special case? Is it possible to do the same for other problems that are known to be Unique-Games-hard but not NP-hard, such as constant-factor approximation of Balanced Separator?
1.1 Our Results
In this work, we answer the above questions affirmatively by designing “fast” exponential time approximation algorithms for Vertex Cover, Uniform Sparsest Cut and related problems. For Vertex Cover, our algorithm gives -approximation in time where is the number of vertices in the input graph and is a parameter that can be any real number at least one (and can depend on ). This improves upon the aforementioned recent algorithm of Bansal [BCL17] which, for a similar approximation ratio, runs in time . For the remaining problems, our algorithms give -approximation in the same running time, which improves upon a known -approximation algorithms with running time that follow from [CMM10] (see the end of Section 1.2 for more details):
[Main] For any , there is an -time -approximation algorithm for Vertex Cover on -vertex graphs, and, there are -time -approximation algorithms for Uniform Sparsest Cut, Balanced Separator, Min UnCut and Min 2CNF Deletion.
1.2 Other Related Works
To prove Theorem 1.1, we use the Sum-of-Square relaxations of the problems and employ the conditioning framework from [BRS11, RT12] together with the main structural lemma from Arora, Rao and Vazirani’s work [ARV09]. We will describe how these parts fit together in Section 2. Before we do so, let us briefly discuss some related works not yet mentioned.
is a generic yet powerful meta-algorithm that can be utilized to any polynomial optimization problems. The approach has found numerous applications in both continuous and combinatorial optimization problems. Most relevant to our work is the conditioning framework developed in[BRS11, RT12]. Barak [BRS11] used it to provide an algorithm for Unique Games with similar guarantee to [ABS15], while Raghavendra and Tan [RT12] used the technique to give improved approximation algorithms for CSPs with cardinality constraints. A high-level overview of this framework is given in Sections 2.2 and 2.3.
Approximability of Vertex Cover, Sparsest Cut and Related Problems. All problems studied in our work are very well studied in the field of approximation algorithms and hardness of approximation. For Vertex Cover, the greedy 2-approximation algorithm has been known since the 70’s (see e.g. [GJ79]). Better -approximation algorithms were independently discovered in [BYE85] and [MS85]. These were finally improved by Karakostas [Kar09] who used the ARV Structural Theorem to provide a -approximation for the problem. On the lower bound side, Hastad [Hås01] show that -approximation for Vertex Cover is NP-hard. The ratio was improved in [DS05] to 1.36. The line of works that very recently obtained the proof of the (imperfect) 2-to-1 game conjecture [KMS17, DKK16, DKK17, KMS18] also yield NP-hardness of -approximate Vertex Cover as a byproduct. On the other hand, the Unique Games Conjecture (UGC) [Kho02] implies that approximating Vertex Cover to within a factor is NP-hard [KR08, BK09]. We remark here that only Hastad reduction (together with Moshkovitz-Raz PCP [MR10]) implies an almost exponential lower bond in terms of the running time, assuming ETH. Putting it differently, it could be the case that Vertex Cover can be approximated to within a factor in time say , without refuting any complexity conjectures or hypotheses mentioned here. Indeed, the question of whether a subexponential time -approximation algorithm for Vertex Cover exists for some constant was listed as an “interesting” open question in [ABS15], and it remains so even after our work.
As for (Uniform) Sparsest Cut and Balanced Separator, they were both studied by Leighton and Rao who gave -approximation algorithms for the problems [LR99]. The ratio was improved in [ARV09] to . In terms of hardness of approximation, these problems are not known to be NP-hard or even UGC-hard to approximate to even just 1.001 factor. (In contrast, the non-uniform versions of both problems are hard to approximate under UGC [CKK06, KV15].) Fortunately, inapproximability results of Sparsest Cut and Balanced Separator are known under stronger assumptions [Fei02, Kho06, RST12]. Specifically, Raghavendra [RST12] shows that both problems are hard to approximate to any constant factor under the Small Set Expansion Hypothesis (SSEH) [RS10]. While it is not known whether SSEH follows from UGC, they are similar in many aspects, and indeed subexponential time algorithms for Unique Games [ABS15, BRS11] also work for the Small Set Expansion problem. This means, for example, that there could be an -approximation algorithm for both problems in subexponential time without contradicting with any of the conjectures. Whether such algorithm exists remains an intriguing open question.
Finally, both Min UnCut and Min 2CNF Deletion are shown to be approximable to within a factor of in polynomial time by Agarwal [ACMM05], which improves upon previous known -approximation algorithm for Min UnCut and -approximation algorithm for Min 2CNF Deletion by Garg [GVY96] and Klein [KPRT97] respectively. On the hardness side, both problems are known to be NP-hard to approximate to within factor for some [PY91]. Furthermore, both are UGC-hard to approximate to within any constant factor [KKMO07, CKK06, KV15]. That is, the situations for both problems are quite similar to Sparsest Cut and Balanced Separator: it is still open whether there are subexponential time algorithms that yield -approximation for Min UnCut and Min 2CNF Deletion.
Fast Exponential Time Approximation Algorithms. As mentioned earlier, Bansal [BCL17] recently gave a “better than brute force” approximation algorithm for Vertex Cover. Their technique is to first observe that we can use branch-and-bound on the high-degree vertices; once only the low-degree vertices are left, they use Halperin’s (polynomial time) approximation algorithm for Vertex Cover on bounded degree graphs [Hal02] to obtain a good approximation. This approach is totally different than ours, and, given that the only way known to obtain -approximation in polynomial time is via the ARV Theorem, it is unlikely that their approach can be improved to achieve similar trade-off as ours.
[BCL17] is not the first work that gives exponential time approximation algorithms for Vertex Cover. Prior to their work, Bourgeois [BEP11] gives a -approximation -time algorithm for Vertex Cover; this is indeed a certain variant of the “limited brute force” algorithm. Furthermore, Bansal [BCL17] remarked in their manuscript that Williams and Yu have also independently come up with algorithms with similar guarantees to theirs, but, to the best of our knowledge, Williams and Yu’s work is not yet made publicly available.
For Sparsest Cut, Balanced Separator, Min UnCut and Min 2CNF Deletion, it is possible to derive -approximation algorithms that run in -time from a work of Charikar [CMM10]. In particular, it was shown in [CMM10] that, for any metric space of elements, if every subset of elements can be embedded isometrically into , then the whole space can be embedded into with distortion . Since -level of Sherali-Adams (SA) relaxations for these problems ensure that every -size subset of the corresponding distance metric space can be embedded isometrically into , -level of SA relaxations, which can be solved in time, ensures that the entire metric space can be embedded into with distortion . An algorithm with approximation ratio can be derived from here, by following the corresponding polynomial time algorithm for each of the problems ([LR99, Kar09, ACMM05]).
In the next section, we describe the overview of our algorithms. Then, in Section 3, we formalize the notations and state some preliminaries. The main lemma regarding conditioned SoS solution and its structure is proved in Section 4. This lemma is subsequently used in all our algorithms which are presented in Section 5. We conclude our paper with several open questions in Section 6.
2 Overview of Technique
Our algorithms follow the “conditioning” framework developed in [BRS11, RT12]. In fact, our algorithms are very simple provided the tools from this line of work, and the ARV structural theorem from [ARV09, Lee05]. To describe the ideas behind our algorithm, we will first briefly explains the ARV structural theorem and how conditioning works with Sum-of-Squares hierarchy in the next two subsections. Then, in the final subsection of this section, we describe the main insight behind our algorithms. For the ease of explaining the main ideas, we will sometimes be informal in this section; all algorithms and proofs will be formalized in the sequel.
For concreteness, we will use the -Balanced Separator problem as the running example in this section. In the -Balanced Separator problem, we are given a graph and the goal is to find a partition of into and that minimizes the number of edges across the cut while also ensuring that for some constant where . Note that the approximation ratio is the ratio between the number of edges cut by the solution and the optimal under the condition . (That is, this is a pseudo approximation rather than a true approximation.) For the purpose of exposition, we focus only on the case where .
2.1 The ARV Structural Theorem
The geometric relaxation used in [ARV09] embeds each vertex into a point such that . For a partition , the intended solution is if and otherwise, where
is some unit vector. As a result, the objective function here is, and the cardinality condition is enforced by . Furthermore, Arora et al. [ARV09] also employ the triangle inequality: for all . In other words, this relaxation can be written as follows.
Note here that the above relaxation can be phrased as a semidefinite program and hence can be solved to arbitrarily accuracy in polynomial time. The key insight shown by Arora is that, given a solution to the above problem, one can find two sets of vertices that are apart from each other, as stated below. Note that this version is in fact from [Lee05]; the original theorem of [ARV09] has a worst parameter with .
[ARV Structural Theorem [ARV09, Lee05]] Let be any vectors in satisfying (2), (3), (4). There exist disjoint sets each of size such that, for every and , . Moreover, such sets can be found in randomized polynomial time.
It should be noted that, given the above theorem, it is easy to arrive at the -approximation algorithm for balanced separator. In particular, we can pick a number uniformly at random from and then output and
. It is easy to check that the probability that each edgeis cut is at most . Moreover, we have and , meaning that we have arrived at an -approximate solution for Balanced Separator.
An interesting aspect of the proof of [Lee05] is that the bound on can be improved if the solution is “hollow” in the following sense: for every , the ball of radius111Here 0.1 can be changed to arbitrary positive constant; we only use it to avoid introducing additional parameters. 0.1 around contains few other vectors ’s. In particular, if there are only such ’s, then can be made , instead of in the above version. We will indeed use this more fine-grained version (in a black-box manner) in our algorithms. To the best of our knowledge, this version of the theorem has not yet been used in other applications of the ARV Structural Theorem.
2.2 Conditioning in Sum-of-Square Hierarchies
Another crucial tool used in our algorithm is Sum-of-Square hierarchy and the conditioning technique developed in [BRS11, RT12]. Perhaps the most natural interpretation of the sum-of-square solution with respect to the conditioning operation is to view the solution as local distributions. One can think of a degree- sum-of-square solution for Balanced Separator as a collection of local distributions over for subsets of vertices of sizes at most that satisfies certain consistency and positive semi-definiteness conditions, and additional linear constraints corresponding to and the triangle inequalities. More specifically, for every and every , the degree- sum-of-squares solution gives us which is a number between zero and one. The consistency constraints ensures that these distributions are locally consistent; that is, for every , the marginal distribution of on is equal to . We remark here that, for Balanced Separator and other problems considered in this work, a solution to the degree- SoS relaxation for them can be found in time .
This consistency constraint on these local distributions allow us to define conditioning on local distributions in the same ways as typical conditional distributions. For instance, we can condition on the event if ; this results in local distributions where is the conditional distribution of on the event . In other words, for all ,
Notice that the local distributions are now on subsets of at most vertices instead of on subsets of at most vertices. In other words, the conditioned solution is a degree- solution.
As for the semi-definiteness constraint, it suffices for the purpose of this discussion to think about only the degree-2 solution case. For this case, the semi-definiteness constraint in fact yields unit vectors such that
It is useful to also note that the probability that are on different side of the cut is exactly equal to ; this is just because
are boolean random variables such thatand .
2.3 Our Algorithms: Combining Conditioning and the ARV Theorem
The conditioning framework initiated in [BRS11, RT12] (and subsequently used in [ABG13, YZ14, MR16]) typically proceeds as follows: solve for a solution to a degree- Sum-of-Square relaxation of the problem for a carefully chosen value of , use (less than ) conditionings to make a solution into an “easy-to-round” degree- solution, and finally round such a solution.
To try to apply this with the Balanced Separator problem, we first have to understand what are the “easy-to-round” solutions for the ARV relaxation. In this regards, first observe that, due to the more refined version of the ARV Theorem (Theorem 2.1), the approximation ratio is actually which can be much better than . In particular, if , this already yields the desired -approximation algorithm. This will be one of the “easy-to-round” situations. Observe also that we can in fact relax the requirement even further: it suffices if holds for a constant fraction of vertices . This is because we can apply Theorem 2.1 on only the set of such ’s which would still result in well-separated set of size . Recall also that from (5) the condition is equivalent to .
Another type of easy-to-round situation is when, for most (i.e. ) of , . In this latter scenario, we can simply find a pair of large well-separated sets by just letting and . It is not hard to argue that both are at least and that, for every and , is at least .
To recap, it suffices for us to condition degree- solution so that we end up in one of the following two “easy-to-round” cases in order to get approximation algorithm for the problem.
For at least vertices , we have .
For at least vertices , we have .
Here we will pick our to be ; the running time needed to solve for such a solution is indeed as claimed. Now, suppose that we have a degree- solution that does not belong to any of the two easy-to-round cases as stated above. This means that there must be such that and that . For simplicity, let us also assume for now that . We will condition on the event ; let the local distributions after conditioning be . Consider each such that . Observe first that, before the conditioning, we have
On the other hand, after the conditioning, we have
Thus, this conditioning makes at least vertices ’s such that beforehand satisfy afterwards. If we ignore how conditioning affects the remaining variables for now, this means that, after such conditioning all vertices must have . Hence, we have arrived at an “easy-to-round” solution and we are done! The effect to the other variables that we ignored can easily be taken into account via a simple potential function argument and by considering conditioning on both and ; this part of the argument can be found in Section 4. This concludes the overview of our algorithm.
3.1 Sum-of-Square Hierarchy, Pseudo-Distribution, and Conditioning
We define several notations regarding the Sum-of-Square (SoS) Hierarchy; these notations are based mainly on [BBH12, OZ13]. We will only state preliminaries necessary for our algorithms. We recommend interested readers to refer to [OZ13, BS14] for a more thorough survey on SoS.
We use to denote the set of all polynomials on of total degree at most . First, we define the notion of pseudo-expectation, which represents solutions to SoS Hierarchy:
[Pseudo-Expectation] A degree- pseudo-expectation is a linear operator that satisfies the following:
(Linearity) For any and , .
(Positivity) For any , .
Furthermore, is said to be boolean if for all .
Observe that, while is a function over infinite domain, has a succinct representation: due to its linearity, it suffices to specify the values of all monomials of total degree at most and there are only such monomials. Furthermore, for boolean , we can save even further since it suffices to specify only products of at most different variables. There are only such terms. From now on, we will only consider boolean pseudo-expectations. Note also that we use as variables instead of variable as used in the proof overview. (Specifically, in the language of the proof overview section, is now equal to .)
A system of polynomial constraints consists of the set of equality constraints and the set of inequality constraints , where all and are polynomials over . We denote the degree of by where denote the (total) degree of polynomial .
For every , we use to denote the monomial . Furthermore, for every and every , let be the polynomial . A boolean degree- pseudo-expectation is said to satisfy a system of polynomial constraints if the following conditions hold:
For all and all such that , we have .
For all , all such that and all , we have .
Note that there are only equalities and inequalities generated above; indeed all degree- SoS relaxations considered in our work can be solved in time since it can be expressed as a semidefinite program222It has been recently pointed out by O’Donnell [O’D17] that the fact that SoS can be written as small SDP is not sufficient to conclude the bound on the running time. However, this is not an issue for us since we are working with the primal solutions (as opposed to sum-of-square certificates) and we can tolerate small errors in each of the equalities and inequalities. In particular, the ellipsoid algorithm can find, in time polynomial of the size of the program, a solution where the error in each inequality is at most say , and this suffices for all of our algorithms. of size .
[Conditioning] Let be any boolean degree- pseudo-expectation for some . For any such that , we denote the conditional pseudo-expectation of on by where
for all .
The proposition below is simple to check, using the identity .
Let be as in Definition 3.1. If satisfies a system of polynomial constraints , then also satisfies the system .
3.2 ARV Structural Theorems
Having defined appropriate notations for SoS, we now move on to another crucial preliminary: the ARV Structural Theorem. It will be useful to state the theorem both in terms of metrics and in terms of pseudo-expectation. Let us start by definitions of several notations for metrics.
[Metric-Related Notations] A metric on is a distance function that satisfies333Here we do not require “identity of indiscernibles ” (i.e. if and only if ), which is sometimes an axiom for metrics in literature. Without such a requirement, is sometimes referred to as a pseudometric. (1) , (2) symmetry and (3) triangle inequality , for all . We use the following notations throughout this work:
For and , and .
We say that are -separated iff .
The diameter of a metric space denoted by is .
We say that is -spread if .
An (open) ball of radius around denoted by is defined as .
A metric space is said to be (, )-hollow if for all .
[Negative Type Metric] A metric space is said to be of negative type if is Euclidean. That is, there exists such that for all .
The ARV Theorem states that, in any negative type metric space that is -spread and -hollow, there exists two large subsets that are -separated:
[ARV Structural Theorem - Metric Formulation [ARV09, Lee05]] Let be any positive real number and be any positive integer. For any negative type metric space with that is -spread and -hollow, there exist disjoints subsets each of size such that . Moreover, these sets can be found in randomized polynomial time.
We remark that the quantitative bound comes from Lee’s version of the theorem [Lee05] whereas the original version only have . We also note that even Lee’s version of the theorem is not stated exactly in the above form; in particular, he only states the theorem with , for which the Hollowness condition is trivial. We will neither retread his whole argument nor define all notations from his work here, but we would like to point out that it is simple to see that his proof implies the version that we use as well. Specifically, the inductive hypothesis in the proof of Lemma 4.2 of [Lee05] implies that when the procedure fails (with constant probability) to find that are separated by where is sufficiently large, then there exists that is -covered by . Lemma 4.1 of [Lee05] then implies that, for each , we must have .
As we are using the ARV Theorem in conjunction with the SoS conditioning framework, it is useful to also state the theorem in SoS-based notations. To do so, let us first state the following fact, which can be easily seen via the fact that the moment matrix (with-entry equal to ) is positive semidefinite and thus is a Gram matrix for some set of vectors:
Let be any degree-2 pseudo-expectation that satisfies the triangle inequality for all . Define by . Then, is a negative type metric space.
When it is clear which pseudo-expectation we are referring to, we may drop the subscript from and simply write . Further, we use all metric terminologies with in the natural manner; for instance, we say that are -separated if .
Theorem 3.2 can now be restated in pseudo-expectation notations as follows.
(Boolean) For every , .
(Triangle Inequality) For every , .
(Hollowness) For all , .
Then, there exists a randomized polynomial time algorithm that, with probability 2/3, produces disjoint subsets each of size at least such that are -separated for .
Notice that, for boolean , . This means that is simply . Another point to notice is that the metric can have as large as 4, instead of 1 required in Theorem 3.2, but this poses no issue since we can scale all distances down by a factor of 4.
We also need a slight variant of the theorem that does not require the balanceness constraint; such variant appears in [Kar09, ACMM05]. It is proved via the “antipodal trick” where, for every , one also add an additional variable and add the constraint to the system. Applying the above lemma together with an observation that the procedure to creates a set from [ARV09] can be modified so that iff gives the following:
[ARV Structural Theorem for Antipodal Vectors [Kar09]] Let be any degree-2 pseudo-expectation that satisfies the following conditions for any and :
(Boolean) For every , .
(Triangle Inequality) For every ,
(Hollowness) For all , .
Then, there exists a randomized polynomial time algorithm that, with probability 2/3, produces disjoint subsets such that and, for every and , we have .
3.3 The Problems
The following are the list of problems we consider in this work.
Vertex Cover. A subset of vertices is said to be a vertex cover of if, for every edge , contains at least one of or . The goal of the vertex cover problem is to find a vertex cover of minimum size.
Sparsest Cut. Given a graph . The (edge) expansion of is defined as , where denote the set of edges across the cut . In the uniform sparsest cut problem, we are asked to find a subset of vertices that minimizes .
Balanced Separator. In the Balanced Separator problem, the input is a graph and the goal is to find a partition of into with for some constant such that is minimized. Note that the approximation ratio is with respect to the minimum for all partition such that where is some constant greater than . In other words, the algorithm is a pseudo (aka bi-criteria) approximation; this is also the notion used in [LR99, ARV09].
For simplicity, we only consider the case where in this work; it is easy to see that the algorithm provided below can be extended to work for any constant .
Minimum UnCut. Given a graph , the Minimum UnCut problem asks for a subset of vertices that minimizes the number of edges that do not cross the cut .
Minimum 2CNF Deletion. In this problem, we are given a 2CNF formula and the goal is to find a minimum number of clauses such that, when they are removed, the formula becomes satisfiable. Here we use to denote the number of variables in the input formula.
4 Conditioning Yields Easy-To-Round Solution
The main result of this section is the following lemma on structure of conditioned solution:
Let be any positive real numbers such that . Given a boolean degree- pseudo-expectation for a system and an integer , we can, in time , find a boolean degree- pseudo-expectation for the system such that the following condition holds:
Let denote the set of indices of variables whose pseudo-expectation lies in and, for each , let denote the set of all indices ’s such that