Semidefinite programming is the problem of optimizing a linear function over a convex set described by a linear matrix inequality:
where has the form:
Here are real symmetric matrices of size and the notation indicates that is positive semidefinite. A set of the form (1) is called a spectrahedron.
Given a convex set , we say that has a semidefinite lift of size if it can be expressed as
where is a spectrahedron (1) defined using matrices of size and is any linear map. If can be expressed in this way, then any linear optimization problem over can be expressed as a semidefinite program of size . The size of the smallest semidefinite lift of is called the positive semidefinite rank of [GPT13, FGP15].
The purpose of this paper is to give a general lower bound on the positive semidefinite rank of convex bodies. Here, by a convex body we mean a closed convex set such that the origin lies in the interior of . For the statement of our main theorem, we need the notion of polar of a convex body , defined as follows:
The polar of a convex body is another full-dimensional closed convex set that is bounded and contains the origin [Roc97, Theorem 14.6]. Throughout the paper, we use for the logarithm base 2. We can now state the first main result of this article.
Let be a convex body and let be its polar. Let be the smallest degree of a polynomial with real coefficients that vanishes on the boundary of . Then .
We also show that this bound is tight in general (up to multiplicative factors):
There exist convex bodies where the degree of the algebraic boundary of can be made arbitrary large and where .
on the size of linear programming lifts. Thelinear programming extension complexity of a polytope is the smallest such that can be written as the linear projection of a polytope with facets.
Theorem 3 (Goemans [Goe15, Theorem 1]).
Assume is a polytope with vertices. Then the linear programming extension complexity of is at least .
The proof is elementary so we include it for completeness. Assume where is a polytope with facets. The pre-image by of any vertex of is a face of . Since has at most faces it follows that . ∎
For functions we say that if there exists a constant such that for all large enough .
The only previous lower bound on the positive semidefinite rank that applies to arbitrary convex bodies that we are aware of is the bound proved in [GPT13, Proposition 6] which says that333In the bound shown in [GPT13, Proposition 6], is the degree of the algebraic boundary of . However since it can also be taken to be that of in the statement of the lower bound. where is the dimension of
. This bound was obtained using results from quantifier elimination theory and one drawback is that it involves constants that are unknown or difficult to estimate. Our lower bound of Theorem1 improves on this existing bound and also has the advantage of being explicit.
The main idea behind the proof of Theorem 1 is simple. Given a convex body , we exhibit a system of polynomial equations that vanishes on the boundary of . This system of polynomial equations is nothing but the Karush-Kuhn-Tucker (KKT) system, after discarding the inequality constraints to get an algebraic variety. Applying the Bézout theorem on the KKT system gives us an upper bound on the degree of this variety and yields the stated lower bound. To prove Theorem 2 about tightness of the bound we appeal to existing works [NRS10] where the degree of the KKT system was explicitly computed, under certain genericity assumptions. The convex bodies of Theorem 2 are in fact random spectrahedra (i.e., spectrahedra defined using random matrices ) of appropriate dimension, where the formulas for the algebraic degree of semidefinite programming [vBR09] allow us to lower bound the degree of the algebraic boundary of their polars. We would like to point out that many of the ideas involved in the proofs of Theorems 1 and 2 appear in some form or another in [RS12, NRS10, SS15]. For example a study of the algebraic boundary of polars of spectrahedra appears in [RS12, Section 5.5]. However it seems that the connection with the positive semidefinite rank was not made explicit before. The focus in these previous works seemed to be on getting exact values for the degrees, at the price of genericity assumptions. In the present work our aim was on getting bounds (tight up to constant factors) but valid without any genericity assumption.
The (topological) boundary of a set is denoted and defined as where and denote closure and interior respectively. The algebraic boundary of denoted is the smallest affine algebraic variety in that contains . We denote by the space of
real symmetric matrices. This is a real vector space of dimension
We also denote by the space of symmetric matrices with complex entries.
2 Proof of Theorem 1
In this section we prove Theorem 1. To do so we will exhibit polynomial equations that vanish on the boundary of polars of spectrahedra and their shadows. These equations are nothing but the KKT conditions of optimality. Applying the Bézout bound will yield Theorem 1.
Let and define
Consider the linear optimization problem
and assume that the feasible set
contains 0 in its interior. In this case we know that any optimal point of (2) must satisfy the following KKT conditions:
where the variable plays the role of dual multiplier and . Conditions (3) consist of equality conditions as well as inequality conditions. If we disregard the inequality conditions we get a system of polynomial equations in which we denote by :444We note here that there are multiple ways of writing the SDP complementarity conditions in general, and these can lead to differences in the context of algorithms for SDP, see e.g., the discussion in [BTN01, Section 6.5.4]. For our purposes, the main property that we will need of the system is that it has a finite number of solutions generically (Lemma 1).
This system has unknowns and consists of equations. A crucial fact about this system is that it has a finite number of solutions, assuming the parameters and are generic (we come back to the genericity assumption after the statement of the result; some form of genericity is needed for the statement to be true). It is the number of solutions to the KKT system that will give an upper bound on the degree of the algebraic boundary of the polar as we will see later.
Lemma 1 (Finiteness of KKT solutions).
For generic and , the KKT system of polynomial equations (4) has a finite number of complex solutions . Furthermore the number of such solutions is at most .
That the KKT system has a finite number of solutions generically was proved in [NRS10, Theorem 7]. We include a sketch of proof for completeness which is simply a dimension count argument. There are three equations in (4):
The equation is linear and defines an affine subspace of codimension (we assume that is injective).
The equation is also linear and defines an affine subspace of codimension .
Finally the equations can be shown to define a variety of codimension (see e.g., [NRS10, Proof of Theorem 7]).
If and are generic, a Bertini-Sard type theorem tells us that the intersection of these three varieties will have codimension equal to the sum of the codimensions, i.e., which is the dimension of the ambient space. In other words the variety defined by (4) is zero-dimensional, i.e., there are a finite number of solutions.
Bézout bound tells us that the number of solutions is at most the product of the degrees of the polynomial equations that form the system (4), which in this case is . ∎
Remark 1 (Genericity assumption of Lemma 1).
An assumption of genericity is necessary in general to guarantee that the system (4) has a finite number of solutions. This is to rule out situations where the optimization problem (2) has an infinite number of solutions (a positive dimensional face of ) or when there are an infinite number of dual multipliers. In Lemma 1 we assumed all the parameters generic to be able to apply a standard Bertini-Sard type theorem. We think however it may be possible to remove some of the genericity assumptions (e.g., just to assume genericity on and ) but we did not pursue this further here as the current statement of the lemma will be sufficient for our purposes.
The next lemma shows that the number of solutions to the KKT system is intimately tied to the degree of the algebraic boundary of the polar .
Consider a spectrahedron where and assume that . Let be its polar defined by
Then there is a polynomial of degree at most with real coefficients that vanishes on the boundary of .
The points on the boundary of are exactly those such that . Consider the system of polynomial equations obtained by adding the equation to the KKT system:
We think of (5) as a system of equations on the variables . If we eliminate the variables we get an algebraic variety in the variables :
By construction this variety contains the boundary of , i.e., . To bound the degree of it thus suffices to count the number of intersections of with a generic line, since and is a hypersurface [Sin15, Corollary 2.8]. We will do this first in the case where are generic. Let generic and consider the line . Since was defined by eliminating variables from (5), we know that if and only if there exist in the solution set of and . By looking at the equations defining this implies that is in the solution set of . Thus the number of intersection points is at most the cardinality of the solution set of , i.e., . We have thus shown that is a hypersurface of degree at most .
It thus remains to treat the case where in the definition of are not generic. This can be done by using a simple perturbation argument. Let be the total number of the entries in symmetric matrices. Hence, the sequence of matrices represents a point in . For any , there exists a point in in the ball centered at of radius which is generic and represents a sequence of symmetric matrices . Since, by assumption , is positive definite, one can assume w.l.o.g. that is positive definite. Hence the spectrahedra defined by are generic, non-empty and such that . Hence, one can apply to them the above paragraph.
Now, let be a sequence of polynomials of degree at most that vanish on the boundary of . We can rescale each to be unit-normed and we can thus assume that has a convergent subsequence that converges to some polynomial . Clearly the degree of is at most . Finally it is easy to verify that vanishes on the boundary of . ∎
We are now in position to prove Theorem 1 on the lower bound for the positive semidefinite rank. The main idea is that if where is a spectrahedron, then by duality is the intersection of with an affine subspace and thus the algebraic boundary of has degree at most that of .
Proof of Theorem 1.
Assume is a convex body that can be written as where is a spectrahedron defined using an linear matrix inequality and a linear map. We can assume that has nonempty interior, and furthermore that since . We are going to exhibit a polynomial of degree at most that vanishes on the boundary of . Let be a polynomial of degree at most that vanishes on the boundary of . Then we claim that the polynomial (where is the adjoint of ), which has degree at most vanishes on the boundary of . Indeed if is on the boundary of this means that which means that and so is on the boundary of , hence .
If we let be the degree of the algebraic boundary of and we have thus shown that which implies . ∎
Application: number of vertices of spectrahedral shadows.
In this subsection we discuss an application of Theorem 1 to bound the number of vertices of spectrahedral shadows. If is a convex body and , the normal cone of at is defined as
A point is called a vertex if is full-dimensional. Observe that any vertex of must be an extreme point, but not all extreme points are vertices, see Figure 1. Vertices play the role of singularities on the boundaries of convex sets; in fact they are also sometimes called 0-singular points. It is known, see e.g., [Sch13, Theorem 2.2.5]
that any convex set has at most a countable number of vertices. Vertices of spectrahedra arising from combinatorial optimization problems have been studied in[LP95, dCST15]. The next theorem gives an upper bound on the number of vertices of any spectrahedral shadow. To the best of our knowledge this is the first such bound.
If is a convex body having a semidefinite representation of size , then has at most vertices.
Any vertex of will contribute a linear factor in the algebraic boundary of : indeed if is a vertex of then the algebraic boundary of
must contain the hyperplane(see e.g., Figure 1(right)). Thus the degree of is greater than or equal the number of vertices of . The result follows since the degree of is at most . ∎
3 Tightness of lower bound, and algebraic degree of semidefinite programming
Let be a spectrahedron defined using matrices . In the previous section we saw that if we project the following KKT equations
on we get an algebraic variety
that vanishes on the boundary of . This variety could coincide exactly with but it can also contain spurious components that do not intersect and thus are not in its Zariski closure (see Section 4 later for an example).
In order to prove our result we need to understand the irreducible components of the variety . If we can show that there is an irreducible component of whose intersection with the boundary of has dimension the one of then we know that the degree of the algebraic boundary of is at least . When is generic, the irreducible components of have been studied in [NRS10] where it was shown that they are obtained by imposing rank conditions on the matrices and in the KKT equations, namely by considering the following system for a fixed :
We think of (8) as a system of equations in . If we eliminate the variables from the above equations we get an algebraic variety in that is contained in . We call this variety :
For generic , it was shown in [NRS10, Theorem 13] that is a hypersurface provided satisfies the Pataki bounds:
Using Bertini theorem one can show that this variety is also irreducible over provided .
For generic the variety is irreducible over provided .
Before proving this lemma we first explain the reason for the condition (which is stronger than the condition imposed by the Pataki bound (10)). The variety is the dual of the determinantal variety . The condition rules out the case where this determinantal variety is zero-dimensional, in which case the dual variety is a union of hyperplanes and is thus not irreducible. Note that if we are only interested in irreducibility statements over (assuming that are generic with entries in ) then we do not need to impose such a condition. See [SS15, Remark 2.2] for more on this.
Proof of Lemma 3.
The main ingredient of the proof is Bertini’s irreducibility theorem [Deb99, Theorem 4.23]. We will start by showing that the variety
is irreducible for a generic choice of . In [NRS10, Lemma 6] it was shown that is irreducible. Consider the projection map . We know that is the determinantal variety consisting of symmetric matrices of rank and has dimension . By Bertini theorem [Deb99, Theorem 4.23] we know that for a generic affine subspace of dimension the variety is going to be irreducible provided , i.e., provided that . In other words this tells us that (11) is irreducible for a generic choice of .
Consider now the map (where the last coordinates stand for ). Observe that the image of the restriction of to the solution set of (11) is exactly the variety defined by (8). Since is rational at all points, it is regular [Sha77, Thm 4, Sec. 3.2]. Because the solution set of (11) is irreducible, its image by is irreducible. Since is the projection of an irreducible variety it is also irreducible. ∎
The degrees of the irreducible components were computed (for generic ) in [NRS10, vBR09] and are denoted by . The resulting formulas involve minors of the matrix of binomial coefficients. An elementary analysis of these formulas allows us to show that in a special regime for and , the algebraic degree is at least .
Assume even and large enough and consider and . Then for generic the variety has degree .
The proof is in Appendix A. ∎
In order to use Lemma 4 we need to show that there is at least one choice of with such that the variety with will actually belong to , where . We can prove this by appealing to results by Amelunxen and Bürgisser [AB15]
where random semidefinite programs were analyzed and where it was shown that every value of rank in the Pataki range occurs with “positive probability”.
Let and be fixed. Let in the associated Pataki range (10) with the additional constraint . Let be any Zariski open set in . Then there exists such that the variety is contained in where .
See Appendix B. ∎
The proof of Theorem 2 is now complete:
Consider the following linear matrix inequality:
One can show that the projection of the associated spectrahedron on the variables is the regular pentagon in the plane, i.e., if we let be the spectrahedron associated to and then:
It is not difficult to see that the polar of is another regular pentagon but slightly rotated and scaled:
We see that the variety contains the algebraic boundary of the polar (red lines). However we also see that it has extra components that are not in : these extra components are shown in blue in Figure 2.
The algebraic argument given in this paper can also be used to lower bound the size of second-order cone lifts, or more generally lifts using products of . More precisely one can show that if has a lift using copies of then where is the degree of the algebraic boundary of . In particular we recover the result of Goemans (Theorem 3) with .
There are a couple of questions that we believe it would be interesting to pursue further:
Polytopes: One important question is to know whether the lower bound can be improved in the case where is a polytope? In particular can the lower bound be improved to in this special case? Recall that if is a polytope then is simply its number of vertices.
Vertices of spectrahedra: A related question is to know whether the bound of on the number of vertices of spectrahedral shadows (Theorem 4) is tight? In words, can we find a spectrahedron (or a spectrahedral shadow) that has vertices? We believe that a natural candidate to try are random spectrahedra of appropriate dimension . Results in [AB15] can be useful for this question.
Explicit example: Thirdly, is there an explicit example of a spectrahedron whose polar has an algebraic boundary of degree ?
Analysis of algebraic degree:
Finally we believe it would be useful to have an (asymptotic) analysis of the formulas for the algebraic degrees of semidefinite programming. For this paper we have used elementary manipulations to show that when and for certain values of then is , but we believe a more complete and systematic study of these quantities can be undertaken. For example we conjecture that in the regime the value of for any in the Pataki range is . Proving such a result would allow us to simplify the proof of Theorem 2 by bypassing the need for Lemma 5 (it suffices to take any generic spectrahedron of dimension say and to observe that at least one of the must belong to ). An analysis of the values of would also allow us to improve the constants in Theorems 1 and 2. For example, where we used the Bézout bound in Lemma 1 one can instead use the quantity (where ranges over the Pataki range) as an upper bound on the number of solutions of the KKT system.
We would like to thank Rainer Sinn for clarifications concerning questions of irreducibility used in Lemma 3, and James Saunderson for comments that helped improve the paper. We would also like to thank anonymous referees for their comments on an earlier version of the paper that was submitted for presentation to the conference MEGA 2017. Finally we would like to thank Xavier Allamigeon and Stéphane Gaubert for organizing the session on Semidefinite Optimization at the PGMO 2016 conference where this project was started. The second author is supported by the ANR JCJC GALOP grant.
Appendix A Proof of Lemma 4: analysis of the formula for the algebraic degree of semidefinite programming
In this subsection we prove Lemma 4 which we restate below.
Lemma (Restatement of Lemma 4).
Assume even and large enough and consider and . Then for generic the variety has degree .
For this we rely on the formula for the algebraic degree of semidefinite programming proved in [vBR09].
Let be the degree of the variety where is a generic pencil . A formula for was given in [vBR09] which we describe now. Let be the (infinite) matrix of binomial coefficients, i.e., for . For define to be the sum of all the minors of . For example if is a singleton we have .
For a generic the algebraic degree of (see Equation (9)) is given by:
where for we denote by the sum of the elements of , and .
The main purpose of this Appendix is to prove the following lower bound on in a special regime.
For all large enough even , and we have .
The bounds we give in this appendix are very crude and are not meant to be optimal. We actually conjecture that in the regime , we have for any in the Pataki range (10).
In order to prove our result we will first analyze the value of on intervals. We will show
For any integers we have .
Proof of Lemma 6.
Consider . Then and . Thus . Using Lemma 7 we get
We now use the fact that and for large enough to get . ∎
It thus remains to prove Lemma 7. We can get the value of on intervals by considering the case in (13). Indeed in this case there is only one set that satisfies the constraints of the summation (13) which is . Since it follows that . On the other hand a simpler formula for was provided in [NRS10, Corollary 15], based on a result by Harris and Tu [HT84]. This tells us that
The formula on the right-hand side can be simplified further using simple manipulations to get
To see why this holds, first use the definition of binomial coefficient to get
Separating the terms in (16) we get
Noting that we see that the second factor in (17) is equal to
Appendix B Proof of Lemma 5: occurrence of each value of rank in the Pataki range
In this Appendix we prove Lemma 5 which we restate here for convenience.
Lemma (Restatement of Lemma 5).
Let and be fixed. Let in the associated Pataki range (10) with the additional constraint . Let be any Zariski open set in . Then there exists a pencil such that the variety is contained in , where .
For convenience in this proof we let, for , denote the associated spectrahedron:
where are standard Gaussian with respect to the Euclidean inner product. In other words, each value in the Pataki range occurs with positive probability. Fix in the Pataki range satisfying and consider
Let and let . To prove our claim we will show that intersects the boundary along a semialgebraic set of dimension . This will prove our claim because if we let be this semialgebraic set we then have on the one hand (where denotes the Zariski closure) and on the other hand , the latter following from the fact that is irreducible of dimension and that since is a semialgebraic set of dimension , see [BCR13, Proposition 2.8.2].
It remains to show that intersects along a semialgebraic set of dimension . To see why this holds let
By definition of (recall that is defined in terms of rank-constrained KKT equations) we have . Now observe that is a semialgebraic set of dimension : indeed note that has nonempty interior (since it is a semialgebraic set with positive probability, see Lemma 8) and so has dimension since for any and neighborhood of , (because is the boundary of a full-dimensional convex set). This completes the proof. ∎
If is semialgebraic and has positive probability under the standard Gaussian measure, then has nonempty interior.
Any semialgebraic set can be decomposed as a disjoint union of semialgebraic sets that are homeomorphic to (see [BCR13, Theorem 2.3.6]). Since , must have a component that is homeomorphic to and thus has nonempty interior. ∎
- [AB15] Dennis Amelunxen and Peter Bürgisser. Intrinsic volumes of symmetric cones and applications in convex programming. Mathematical Programming, 149(1-2):105–130, 2015.
- [BCR13] Jacek Bochnak, Michel Coste, and Marie-Françoise Roy. Real algebraic geometry, volume 36. Springer Science & Business Media, 2013.
- [BTN01] Aharon Ben-Tal and Arkadi Nemirovski. Lectures on modern convex optimization: analysis, algorithms, and engineering applications. SIAM, 2001.
- [dCST15] Marcel K. de Carli Silva and Levent Tunçel. Vertices of spectrahedra arising from the elliptope, the theta body, and their relatives. SIAM Journal on Optimization, 25(1):295–316, 2015.
- [Deb99] Olivier Debarre. Introduction à la géométrie algébrique. 1999. Available online at http://www.math.ens.fr/~debarre/DEA99.pdf.
- [FGP15] Hamza Fawzi, João Gouveia, Pablo A. Parrilo, Richard Z. Robinson, and Rekha R. Thomas. Positive semidefinite rank. Mathematical Programming, 153(1):133–177, 2015.
- [Goe15] Michel X. Goemans. Smallest compact formulation for the permutahedron. Mathematical Programming, 153(1):5–11, 2015.
- [GPT13] João Gouveia, Pablo A. Parrilo, and Rekha R. Thomas. Lifts of convex sets and cone factorizations. Mathematics of Operations Research, 38(2):248–264, 2013.
Joe Harris and Loring W. Tu.
On symmetric and skew-symmetric determinantal varieties.Topology, 23(1):71–84, 1984.
- [LP95] Monique Laurent and Svatopluk Poljak. On a positive semidefinite relaxation of the cut polytope. Linear Algebra and its Applications, 223:439–461, 1995.
- [NRS10] Jiawang Nie, Kristian Ranestad, and Bernd Sturmfels. The algebraic degree of semidefinite programming. Mathematical Programming, 122(2):379–405, 2010.
- [Ran12] Kristian Ranestad. Algebraic degree in semidefinite and polynomial optimization. In Handbook on Semidefinite, Conic and Polynomial Optimization, pages 61–75. Springer, 2012.
- [Roc97] R Tyrell Rockafellar. Convex analysis, volume 28. Princeton University Press, 1997.
- [RS12] Philipp Rostalski and Bernd Sturmfels. Dualities in convex algebraic geometry. In Grigoriy Blekherman, Pablo A. Parrilo, and Rekha R. Thomas, editors, Semidefinite Optimization and Convex Algebraic Geometry, chapter 5, pages 203–249. SIAM, 2012.
- [Sch13] Rolf Schneider. Convex bodies: the Brunn–Minkowski theory. Number 151. Cambridge University Press, 2013.
- [Sha77] Igor Shafarevich. Basic Algebraic Geometry 1. Springer Verlag, 1977.
- [Sin15] Rainer Sinn. Algebraic boundaries of convex semi-algebraic sets. Research in the Mathematical Sciences, 2(1):1, 2015.
- [SS15] Rainer Sinn and Bernd Sturmfels. Generic spectrahedral shadows. SIAM Journal on Optimization, 25(2):1209–1220, 2015.
- [vBR09] Hans-Christian Graf von Bothmer and Kristian Ranestad. A general formula for the algebraic degree in semidefinite programming. Bulletin of the London Mathematical Society, 41(2):193–197, 2009.