 # Extension Complexity of the Correlation Polytope

We prove that for every n-vertex graph G, the extension complexity of the correlation polytope of G is 2^O(tw(G) + n), where tw(G) is the tree-width of G. We also prove that this bound is tight for graphs contained in minor-closed classes.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1. Introduction

All graphs in this paper are undirected and simple. Let be a graph. The correlation polytope of , denoted

, is the convex hull of the characteristic vectors of induced subgraphs of

. More precisely, is the polytope in which is the convex hull of all vectors of the form for , where is the set of edges with both endpoints in , and denotes the characteristic vector of a set. It is closely related to the stable set polytope of , denoted , which is the convex hull of characteristic vectors of stable sets of

. There are other equivalent definitions of the correlation polytope, and it arises naturally in many different contexts, including probability theory, propositional logic, and quantum mechanics

.

More recently, the correlation polytope has also acquired greater prominence in machine learning, where it is more commonly referred to as the

marginal polytope. For example, Wainright and Jordan  showed that the maximum a posteriori (MAP) inference problem for graphical models [16, 6, 8, 21] is equivalent to maximizing a linear function over the correlation polytope. This motivates the search for compact descriptions of the correlation polytope. The proper framework for addressing such questions is the theory of extended formulations [11, 4, 5, 20, 27].

A polytope is an extension of a polytope if there exists an affine map with . The extension complexity of , denoted , is the minimum number of facets of any extension of . If is an extension of such that has fewer facets than , then it can be advantageous to optimize over instead of .

Fiorini, Massar, Pokutta, Tiwary and de Wolf  were the first to show that many polytopes associated to classic -hard problems, (including the correlation polytope of the complete graph) have exponential extension complexity. Their results do not rely on any complexity assumptions such as or . In this paper, we determine the extension complexity of the correlation polytope almost exactly.

In order to state our main results, we need some graph theoretic definitions. A graph is a minor of a graph , denoted , if can be obtained from a subgraph of by contracting edges. A class of graphs is minor-closed if and implies . In Observation 5, we note that if , then . By the graph minor theorem of Robertson and Seymour , the property is characterized by a finite set of forbidden minors. For more on the connection between the correlation polytope and graph minors, see [25, 26].

A tree-decomposition of a graph is a pair where is a tree and is a collection of subsets of vertices of satisfying:

• ,

• for each , there exists such that , and

• for each , the set of all such that induces a subtree of .

The width of the decomposition is . The treewidth of , denoted , is the minimum width taken over all tree-decompositions of .

Wainright and Jordan  proved that for all graphs , the -th level of the Sherali-Adams hierarchy  of the relaxation of is equal to . It follows that has an extended formulation of size . We now derive an improved upper bound using results of Kolman and Koutecký  (see also Bienstock and Munoz ).

###### Theorem 1.

For every -vertex graph , the extension complexity of is .

###### Proof.

Let be a graph with vertices and edges. Note that is the convex hull of all -vectors satisfying for all . We define the constraint graph of to be the graph whose vertices are the variables of the above system, where two variables are adjacent in if and only if they appear in a common constraint. By [14, Theorem 1], has an extended formulation of size . Observe that is obtained from by adding a path of length between and for all . It is easy to see that if , then and if , then . Therefore, , as required. ∎

Up to the constant factor in the exponent, we conjecture that our upper bound is tight.

###### Conjecture 2.

For every -vertex graph , the extension complexity of is .

Göös, Jain and Watson  proved that there exists a sequence of graphs where each is an -vertex graph and the extension complexity of is . If true, our conjecture would improve their bound to and it would yield explicit -polytopes with extension complexity exponential in their dimension, as predicted by the counting argument of Rothvoß  (see [27, Problem 7]).

As evidence for the conjecture, we prove that our bound is tight for minor-closed classes of graphs. The following is our main result.

###### Theorem 3.

For every proper minor-closed class , there exists a constant such that for every -vertex graph ,

 xc(COR(G))⩾2c(tw(G)+logn).

Actually, the proof of Theorem 3 shows that for every graph , we have , where is the maximum height of a grid that contains as a minor. In virtue of the polynomial grid-minor theorem of Chekuri and Chuzhoy [2, 3], this implies that for some universal constant .

## 2. The lower bound

In this section we prove our lower bound. Let be a polytope. We note the following easy (and folklore) facts. If is an affine subspace, then , and if is an affine map, then . Since the extension complexity of a polytope is at least its dimension, we also have the following easy observation.

###### Observation 4.

For all -vertex graphs , .

We next show that the extension complexity of the correlation polytope is monotone under taking minors.

If , then .

###### Proof.

It is easy to see that if and only if can be obtained from by deleting edges, contracting edges, and removing isolated vertices. We show that none of these operations increases the extension complexity of the correlation polytope.

Let . Then can be obtained from by projecting out . Moreover is obtained from by setting (this defines a face since and are valid). If is an isolated vertex of , then is obtained from by projecting out . ∎

For , we let be the complete bipartite graph with vertices and edges, and be the grid. Recall that has vertex set , where is adjacent to if and only if .

We now define a modified grid that will appear in the proof of Theorem 6. A grid with gadgets of height is obtained by modifying the grid as follows. Let be the graph obtained from by subdividing each edge of . For each , let be the vertices of along the th row and be the vertices of along the th column. The grid with gadgets of height is obtained from by performing the following operations:

• delete and ,

• delete and ,

• for each , delete the edges and ,

• for each , add the edge ,

• for each , replace by a constant-size planar graph which will be defined in the proof of Theorem 6.

See Figure 1 for an illustration.

We are now ready to prove our lower bound for minor-closed classes.

###### Theorem 6.

For every proper minor-closed class , there exists a constant such that for every -vertex graph ,

 xc(COR(G))⩾2c′tw(G).
###### Proof.

As shown by Demaine and Hajiaghayi  (see also Kawarabayashi and Kobayashi ), since our initial graph belongs to a proper minor-closed class, contains as a minor, where . Moreover, observe that there exists such that if is the grid with gadgets of height , then . Let be the grid with gadgets of height . Since the extension complexity of the correlation polytope is minor monotone, . Therefore, it suffices to show the theorem for .

The idea is to describe a face of which projects to the correlation polytope of the complete bipartite graph . The projection is defined in such a way that the vertices of the bipartition of correspond to bottom and left vertices of the grid with gadgets, and the edges of correspond to the dotted diagonal edges of Figure 1. Roughly speaking, the face is defined in such a way that the value of the variable for each bottom and each left vertex of the grid propagates along the corresponding vertical and horizontal path. The gadgets make sure that propagation along vertical paths does not interfere with propagation along horizontal paths.

The gadget used in the proof is inspired by the crossover gadget of the reduction from 3-SAT to Planar 3-SAT in . In the crossover gadget (see Figure 1(b)) the square vertices represent the clauses of a SAT formula and the round vertices represent the variables. When a round vertex is adjacent to a square via a blue-dashed (resp. red-dotted) edge, this means that the corresponding variable (resp. negation of the variable) appears in the corresponding clause. The main property of the crossover gadget in Figure 1(b) is that if all clauses in the crossover gadget are satisfied, then is true if and only if is true and is true if and only if is true.

We simulate each copy of the crossover gadget with a copy of a fixed planar graph, which we call the graph gadget. Each copy is independent from the other copies: all the new vertices and edges that are created in the process are local to the copy. We ensure that the graph gadget behaves exactly like the crossover gadget by restricting to an appropriate face of . For each variable in the crossover gadget, we interpret as being true or false, according as, the corresponding variable of is equal to or . The grid with gadgets in Figure 1 is obtained by replacing each diamond by the graph gadget.

To convert the crossover gadget into the graph gadget, we describe replacement rules for the neighbourhood of each square vertex (clause). After all the rules are applied, we obtain just a graph. The rules are described in Figures 3 to 7, together with a set of equations that are to be added to the description of the desired face of . The equations we add originate from valid inequalities, hence the set of points of satisfying all of them is indeed a face of .

First, consider Figure 3. It specifies that degree- square vertices adjacent to two red edges should be replaced by a single edge. Here, we want to simulate the clause . Thus, we want at most one of or to be true, which is equivalent to setting .

Next, the replacement described in Figure 4 is to transform paths consisting of one red edge and one blue edge incident to a degree- square vertex into a path of the same length. Here, we want to simulate . We claim this is equivalent to adding the following set of equations: and . The first constraint implies that and do not both equal , while the second constraint ensures that exactly one of and equals . Therefore, at least one of or is true.

Finally, we replace the degree- square vertices as in Figures 5, 6 and 7. Here, we want to simulate the clauses containing three literals. For instance, consider Figure 5, where we want to simulate . The replacements in Figure 6 and 7 are similar. We claim this is described by adding the following set of equations: , , , , and . The last constraint implies that exactly one of , or equals . The second constraint implies that exactly one of and equals . The first constraint implies that at most one of and equals , at most one of and equals , and at most one of and equals . Together, this implies or or , as required.

After these replacements we obtain a constant size graph gadget. The above constraints define a face of such that all clauses inside the crossover gadgets are satisfied. Therefore, by the main property of the crossover gadget (see  for a proof), and (see Figure 2).

If is a vertical or horizontal edge of (see the solid green edges in Figure 1), then we set (note that and are both valid inequalities). Together with the crossover gadgets, this ensures that if is a vertex on the bottom of , then the value of propagates vertically. Similarly, if is a vertex on the left of then the value of propagates horizontally. It follows that by projecting onto the diagonal dotted edges in Figure 1, the face of described above projects to . Finally, since is a minor of , we have . In , it is shown that . (A weaker exponential bound was given earlier in .) Since , this completes the proof. ∎

###### Proof of Theorem 3.

Let be the constant from Theorem 6 and set to be equal to

. By taking the geometric mean of the bounds in Observation

4 and Theorem 6 we obtain Theorem 3. ∎

### Acknowledgements

We thank Mark Rowland and Adrian Weller for interesting discussions on the connection between the correlation polytope and graph minors. We also thank the referee for their careful reading of the paper. This project is supported by ERC grant FOREFRONT (grant agreement no. 615640) funded by the European Research Council under the EU’s 7th Framework Programme (FP7/2007-2013).

## References

•  Daniel Bienstock and Gonzalo Munoz, LP approximations to mixed-integer polynomial optimization problems, 2015.
•  Chandra Chekuri and Julia Chuzhoy, Polynomial bounds for the grid-minor theorem, Journal of the ACM 63 (2016), no. 5, 40:1–40:65.
•  Julia Chuzhoy, Improved bounds for the excluded grid theorem, 2016.
•  Michele Conforti, Gérard Cornuéjols, and Giacomo Zambelli,

Extended formulations in combinatorial optimization

, Ann. Oper. Res. 204 (2013), 97–143. MR 3039264
•  by same author, Integer programming, Graduate Texts in Mathematics, vol. 271, Springer, Cham, 2014. MR 3237726
•  Jasper De Bock, Cassio P De Campos, and Alessandro Antonucci, Global sensitivity analysis for map inference in graphical models, Advances in Neural Information Processing Systems, 2014, pp. 2690–2698.
•  Erik D. Demaine and Mohammadtaghi Hajiaghayi, Linearity of grid minors in treewidth with applications through bidimensionality, Combinatorica 28 (2008), no. 1, 19–36. MR 2399006
•  Murat A Erdogdu, Yash Deshpande, and Andrea Montanari, Inference in graphical models via semidefinite programming hierarchies, Advances in Neural Information Processing Systems, 2017, pp. 416–424.
•  Samuel Fiorini, Serge Massar, Sebastian Pokutta, Hans Raj Tiwary, and Ronald de Wolf, Exponential lower bounds for polytopes in combinatorial optimization, J. ACM 62 (2015), no. 2, Art. 17, 23. MR 3346156
•  Mika Göös, Rahul Jain, and Thomas Watson, Extension complexity of independent set polytopes, SIAM Journal on Computing 47 (2018), no. 1, 241–269.
•  Volker Kaibel, Extended formulations in combinatorial optimization, 2011.
•  Volker Kaibel and Stefan Weltge, A short proof that the extension complexity of the correlation polytope grows exponentially, Discrete & Computational Geometry 53 (2015), no. 2, 397–401.
•  Ken-ichi Kawarabayashi and Yusuke Kobayashi, Linear min-max relation between the treewidth of -minor-free graphs and its largest grid minor, 29th International Symposium on Theoretical Aspects of Computer Science, LIPIcs. Leibniz Int. Proc. Inform., vol. 14, Schloss Dagstuhl. Leibniz-Zent. Inform., Wadern, 2012, pp. 278–289. MR 2909321
•  Petr Kolman and Martin Koutecký, Extended formulation for CSP that is compact for instances of bounded treewidth, Electron. J. Combin. 22 (2015), no. 4, Paper 4.30, 14. MR 3441650
•  David Lichtenstein, Planar formulae and their uses, SIAM journal on computing 11 (1982), no. 2, 329–343.
•  Xianghang Liu, James Petterson, and Tiberio S Caetano, Learning as map inference in discrete graphical models, Advances in Neural Information Processing Systems, 2012, pp. 1970–1978.
•  Itamar Pitowsky, Correlation polytopes: their geometry and complexity, Math. Programming 50 (1991), no. 3, (Ser. A), 395–414. MR 1114239
•  N. Robertson and P.D. Seymour, Graph minors. XX. Wagners conjecture, J. Comb. Theory Ser. B 92 (2004), no. 2, 325–357.
•  Thomas Rothvoß, Some 0/1 polytopes need exponential size extended formulations, Mathematical Programming 142 (2013), no. 1-2, 255–268.
•  Tim Roughgarden, Communication complexity (for algorithm designers), 2015.
•  Mark Rowland and Adrian Weller, Uprooting and rerooting higher-order graphical models, Advances in Neural Information Processing Systems, 2017, pp. 208–217.
•  Hanif D. Sherali and Warren P. Adams, A hierarchy of relaxations between the continuous and convex hull representations for zero-one programming problems, SIAM J. Discrete Math. 3 (1990), no. 3, 411–430. MR 1061981
•  Martin J Wainwright and Michael I Jordan, Treewidth-based conditions for exactness of the Sherali-Adams and Lasserre relaxations, Tech. report, Technical Report 671, University of California, Berkeley, 2004.
•  by same author, Graphical models, exponential families, and variational inference, Foundations and Trends® in Machine Learning 1 (2008), no. 1–2, 1–305.
•  Adrian Weller, Characterizing tightness of LP relaxations by forbidding signed minors

, Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, AUAI Press, 2016, pp. 765–774.

•  Adrian Weller, Mark Rowland, and David Sontag, Tightness of LP relaxations for almost balanced models, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics (Cadiz, Spain) (Arthur Gretton and Christian C. Robert, eds.), Proceedings of Machine Learning Research, vol. 51, PMLR, 09–11 May 2016, pp. 47–55.
•  Stefan Weltge, Sizes of linear descriptions in combinatorial optimization, Ph.D. thesis, Otto-von-Guericke-Universität Magdeburg, Fakultät für Mathematik, 2015.