Optimization over the Boolean Hypercube via Sums of Nonnegative Circuit Polynomials

02/27/2018 ∙ by Mareike Dressler, et al. ∙ IG Farben Haus ETH Zurich Berlin Institute of Technology (Technische Universität Berlin) 0

Various key problems from theoretical computer science can be expressed as polynomial optimization problems over the boolean hypercube. One particularly successful way to prove complexity bounds for these types of problems are based on sums of squares (SOS) as nonnegativity certificates. In this article, we initiate the analysis of optimization problems over the boolean hypercube via a recent, alternative certificate called sums of nonnegative circuit polynomials (SONC). We show that key results for SOS based certificates remain valid: First, for polynomials, which are nonnegative over the n-variate boolean hypercube with constraints of degree d there exists a SONC certificate of degree at most n+d. Second, if there exists a degree d SONC certificate for nonnegativity of a polynomial over the boolean hypercube, then there also exists a short degree d SONC certificate, that includes at most n^O(d) nonnegative circuit polynomials.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

An optimization problem over a boolean hypercube is an -variate (constrained) polynomial optimization problem where the feasibility set is restricted to the vertices of an -dimensional hypercube

subject to

The formulation (1) captures a class of optimization problems, that belong to the core of theoretical computer science. However, it is known that solving the above formulation is NP-hard in general, since one can easily cast, e.g., the Independent Set problem in this framework.

One of the most promising approaches in constructing efficient algorithms is the sum of squares (SOS) hierarchy [GV01, Nes00, Par00, Sho87], also known as Lasserre relaxation [Las01]. The method is based on a Positivstellensatz result [Put93] saying that the polynomial , which is nonnegative over the feasibility set given in (1), can be expressed as a sum of squares times the constraints defining the set. Bounding a maximum degree of a polynomial used in a representation of provides a family of algorithms parametrized by an integer . Finding a degree SOS certificate for nonnegativity of can be performed by solving a semidefinite programming (SDP) formulation of size . Finally, for every (feasible) -variate unconstrained hypercube optimization problem there exists a degree SOS certificate.

On the one hand, the SOS algorithm provide the best available approximation algorithms for a wide variety of optimization problems. For example, the degree 2 SOS for the Independent Set problem implies the Lovász -function [Lov79] and gives the Goemans-Williamson relaxation [GW95] for the Max Cut problem. The ARV algorithm of the Sparsest Cut [ARV09] problem can be captured by SOS of degree 6. Finally, the subexponential time algorithm for Unique Games [ABS10] is implied by a SOS of sublinear degree [BRS11, GS11]. More recently, it has been shown that degree SOS is equivalent in power to any polynomial size SDP extended formulation in approximating maximum constraint satisfaction problems [LRS15]

. Other applications of the SOS method for combinatorial optimization can be found in 

[BRS11, BCG09, Chl07, CS08, CGM13, dlVKM07, GS11, Mas17, MM09, RT12]. For a more detailed overview on the use of SOS in approximation algorithms, see the surveys [CT, Lau03a, Lau09a].

On the other hand, it is known that the SOS algorithm admits certain weaknesses. First, for some hypercube optimization problems the SOS algorithm performs much worse than other known methods. Grigoriev in [Gri01] shows that a degree SOS certificate is needed to detect that the Knapsack instance contains no integer point. Simpler proofs can be found in [GHP02, Lau03b, KLM16]. Other SOS degree lower bounds for Knapsack problems appeared in [Che07, KLM17a]. Another example is the problem of scheduling unit size jobs on a single machine to minimize the number of late jobs. The problem is solvable in polynomial time using the Moore-Hodgson algorithm; an degree SOS algorithm, however, still attains an unbounded integrality gap [KLM17b]. For other important SOS limitations see e.g. [MPW15, BHK16].

Second, it remains open if finding a degree SOS certificate can be performed in time . Indeed, as noted in the recent paper by O’Donnell [O’D17] and further discussed by Raghavendra and Weitz in [RW17] it is not obviously true that the search can be done so efficiently. Namely, even if a small degree SOS certificate exists, the polynomials in the certificate do not have necessarily small coefficients. O’Donnell in [O’D17] gives an example of a polynomial optimization problem that admits a degree 2 SOS certificate, but every degree 2 SOS certificate for this problem has exponential bit complexity. Moreover, in [RW17] the example is modified and cast into a hypercube optimization problem again having a degree 2 SOS certificate, which, however, has super-polynomial bit complexity for certificates up to the degree . For small , this excludes the possibility that known optimization tools used for solving SDP problems like the ellipsoid method [Kha80, GLS88] are able to find a degree certificates in time for optimization problems of the form (1). The above arguments motivate the search of new methods for solving hypercube optimization problems efficiently.

In this article, we initiate an analysis of hypercube optimization problems of the form (1) via sums of nonnegative circuit polynomials (SONC). SONCs are a nonnegativity certificate introduced recently by Iliman and the third author [IdW16], which are independent of sums of squares; see Definition 2.1 and Theorem 2.4 for further details. Similarly as Lasserre relaxation for SOS, a Schmüdgen-like Positivstellensatz yields a converging hierarchy of lower bounds for polynomial optimization problems with compact constraint set; see [DIdW17, Theorem 4.8] and Theorem 2.5. These bounds can be computed via a convex optimization program called relative entropy programming (REP) [DIdW17, Theorem 5.3]. Our main question in this article is:

Can SONC certificates be an alternative for SOS methods for optimization problems over the hypercube?

We answer this question affirmatively in the sense that we prove SONC complexity bounds for (1) analogous to the SOS bounds mentioned above. More specifically, we show:

  1. For every polynomial which is nonnegative over an -variate hypercube with constraints of degree at most there exists a SONC certificate of nonnegativity of degree at most ; see Theorem 4.7 and Corollary 4.8.

  2. If a polynomial admits a degree SONC certificate of nonnegativity over an -variate hypercube, then the polynomial admits also a short degree SONC certificate that includes at most nonnegative circuit polynomials; see Theorem 4.9.

For a discussion and remaining open problems, to turn these results into an efficient algorithms, see Section 2.3 and the end of Section 4.2.

Furthermore, we show some structural properties of SONCs:

  1. We give a simple, constructive example showing that the SONC cone is not closed under multiplication. Subsequently we use this construction to show that the SONC cone is neither closed under taking affine transformations of variables, see Lemma 3.1 and Corollary 3.2.

  2. We address an open problem raised in [DIdW17] asking whether the Schmüdgen-like Positivstellensatz for SONCs (Theorem 2.5) can be improved to an equivalent of Putinar’s Positivstellensatz [Put93]. We answer this question negatively by showing an explicit hypercube optimization example, which provably does not admit a Putinar representation for SONCs; see Theorem 5.1 and the discussion afterwards.

Our article is organized as follows: In Section 2 we introduce the necessary background about SONCs. In Section 3 we show that the SONC cone is closed neither under multiplication nor under affine transformations. In Section 4 we provide our two main results regarding the degree bounds for SONC certificates over the hypercube. In Section 5 we prove the non-existence of an equivalent of Putinar’s Positivstellensatz for SONCs and discuss this result.

Acknowledgements

AK was supported by the Swiss National Science Foundation project PZ00P2174117 “Theory and Applications of Linear and Semidefinite Relaxations for Combinatorial Optimization Problems”. TdW was supported by the DFG grant WO 2206/1-1. This article was finalized while TdW was hosted by the Institut Mittag-Leffler. We thank the institute for its hospitality.

2. Preliminaries

In this section we collect basic notions and statements on sums of nonnegative circuit polynomials (SONC).

Throughout the paper, we use bold letters for vectors, e.g.,

. Let and () be the set of nonnegative (positive) real numbers. Furthermore let be the ring of real -variate polynomials and the set of all -variate polynomials of degree less than or equal to is denoted by . We denote by the set and the sum of binomial coefficients is abbreviated by . Let denote the canonical basis vectors in .

2.1. Sums of Nonnegative Circuit Polynomials

Let be a finite set. In what follows, we consider polynomials supported on . Thus, is of the form with and . A lattice point is called even if it is in and a term is called a monomial square if and even. We denote by the Newton polytope of .

Initially, we introduce the foundation of SONC polynomials, namely circuit polynomials; see also [IdW16]:

Definition 2.1.

A polynomial is called a circuit polynomial if it is of the form

(2.1)

with , exponents , , and coefficients , , such that the following conditions hold:

(C1):

is a simplex with even vertices .

(C2):

The exponent is in the strict interior of . Hence, there exist unique barycentric coordinates relative to the vertices with satisfying

We call the terms the outer terms and the inner term of .

For every circuit polynomial we define the corresponding circuit number as

(2.2)

The first fundamental statement about circuit polynomials is that its nonnegativity is determined by its circuit number and entirely:

Theorem 2.2 ([IdW16], Theorem 3.8).

Let be a circuit polynomial with inner term and let be the corresponding circuit number, as defined in (2.2). Then the following statements are equivalent:

  1. is nonnegative.

  2. and  or   and .

Therefore, expressing a polynomial as a sum of nonnegative circuit polynomials (SONC) is a certificate for the polynomials nonnegativity.

Definition 2.3.

We define for every the set of sums of nonnegative circuit polynomials (SONC) in variables of degree as

We denote by SONC both the set of SONC polynomials and the property of a polynomial to be a sum of nonnegative circuit polynomials.

In what follows let be the cone of nonnegative -variate polynomials of degree at most and be the corresponding cone of sums of squares respectively. An important observation is, that SONC polynomials form a convex cone independent of the SOS cone:

Theorem 2.4 ([IdW16], Proposition 7.2).

is a convex cone satisfying:

  1. for all ,

  2. if and only if ,

  3. for all with .

For further details about the SONC cone see [dW15, IdW16, DIdW17].

2.2. SONC Certificates over a Constrained Set

In [DIdW17, Theorem 4.8], Iliman, the first, and the third author showed that for an arbitrary real polynomial which is strictly positive on a compact, basic closed semialgebraic set there exists a SONC certificate of nonnegativity. Hereinafter we recall this result.

We assume that is given by polynomial inequalities for and is compact. For technical reason we add redundant box constraints for some sufficiently large , which always exists due to our assumption of compactness of ; see [DIdW17] for further details. Hence, we have

(2.3)

In what follows we consider polynomials defined as products of at most of the polynomials and , i.e.,

(2.4)

where . Now we can state:

Theorem 2.5.

Let be real polynomials and be a compact, basic closed semialgebraic set as in (2.3). If on then there exist such that we have an explicit representation of of the following form:

where the are contained in and every is a product as in (2.4).

The central object of interest is the smallest value of and that allows a decomposition as in Theorem 2.5. This motivates the following definition of a degree SONC certificate.

Definition 2.6.

Let such that is positive on the set given in (2.3). Then has a degree SONC certificate if it admits for some the following decomposition:

where the are SONCs, the are products as in (2.4), and

2.3. The Complexity of Finding a Degree SONC Certificate

One can decide nonnegativity for a single given circuit polynomial by solving a system of linear equations. This is due to Theorem 2.2 and the fact that the are unique (and thus trivially nonnegative), since the are affinely independent by construction.

For finding a degree SONC certificate, there are two main bottlenecks that might effect its complexity. The first one is to guarantee the existence of a sufficiently short degree SONC certificate. If the first bottleneck is resolved, then a second one might occur: even if the existence of a short degree SONC certificate is guaranteed, then it is not clear a priori, whether one can search through the space of -variate circuit polynomials of degree at most efficiently, in order to find such a short certificate.

Regarding the first bottleneck, given the fact that a polynomial admits a degree SONC certificate, it is open whether there also exists a degree certificate which consists of a bounded (ideally ) number of components.

The answer to the equivalent question for the SOS degree certificates follows from the fact that a polynomial is SOS if and only if the corresponding matrix of coefficients of size , called the Gram matrix, is positive semidefinite. Since every real, symmetric matrix that is positive semidefinite admits a decomposition , this yields an explicit SOS certificate including at most polynomials squared. For more details we refer the reader to the excellent lecture notes in [BS16].

In this paper we resolve the first bottleneck regarding the existence of short SONC certificates affirmatively. Namely, we show that one can always restrict oneself to SONC certificates including at most nonnegative circuit polynomials, see Section 4.2 for further details.

3. Properties of the SONC cone

In this section we show that the SONC cone is neither closed under multiplication nor under affine transformations. First, we give a constructive proof for the fact that the SONC cone is not closed under multiplication, which is simpler than the initial proof of this fact in [DIdW17, Lemma 4.1]. Second, we use our construction to show that the SONC cone is not closed under affine transformation of variables.

Lemma 3.1.

For every , the SONC cone is not closed under multiplication in the following sense: if , then in general.

Proof.

For every , we construct two SONC polynomials , such that the product is an variate, degree polynomial that is not inside .

Let . We construct the following two polynomials :

First, observe that are nonnegative circuit polynomials, since, in both cases, , , and , thus .

Now consider the polynomial . We show that this polynomial, even though it is nonnegative, is not a SONC polynomial. Note that ; the support of is shown in Figure 1.

Figure 1. The Newton polytope and the support set of with the supports of and in blue ovals.

Assume that , i.e., has a SONC decomposition. This implies that the term has to be an inner term of some nonnegative circuit polynomial in this representation. Such a circuit polynomial necessarily has the terms and as outer terms, that is,

Since the polynomial is indeed nonnegative and, in addition, we cannot choose a smaller constant term to construct . Next, also the term has to be an inner term of some nonnegative circuit polynomial . Since this term again is on the boundary of the only option for such an is: . However, the term has been already used in the above polynomial , which leads to a contradiction, i.e., . Since , the general statement follows. ∎

Hereinafter we show another operation, which behaves differently for SONC than it does for SOS: Similarly as in the case of multiplications, affine transformations also do not preserve the SONC structure. This observation is important for possible degree bounds on SONC certificates, when considering optimization problems over distinct descriptions of the hypercube.

Corollary 3.2.

For every , the SONC cone is not closed under affine transformation of variables.

Proof.

Consider the polynomial . Clearly, the polynomial is a nonnegative circuit polynomial since it is a monomial square, hence . Now consider the following affine transformation of the variables and :

After applying the transformation the polynomial equals the polynomial from the proof of Lemma 3.1 and thus is not inside .

4. An Upper Bound on the Degree of SONC Certificates over the Hypercube

In the previous section we showed that the SONC cone is not closed under taking an affine transformation of variables, Corollary 3.2. Thus, if a polynomial admits a degree SONC certificate proving that it is nonnegative on a given compact semialgebraic set , then it is a priori not clear whether a polynomial , obtained from via an affine transformation of variables, admits a degree SONC certificate of nonnegativity on , too. The degree needed to prove nonnegativity of might be much larger than according to the argumentation in the proof of Corollary 3.2.

In this section we prove that every -variate polynomial which is nonnegative over the boolean hypercube has a degree SONC certificate. Moreover, if the hypercube is additionally constrained with some polynomials of degree at most , then the nonnegative polynomial over such a set has degree SONC certificate. We show this fact for all hypercubes ; see Theorem 4.3 for further details.

Formally, we consider the following setting: We investigate real multivariate polynomials in . For , and , such that let

be a quadratic polynomial with two distinct real roots. Let denote the -dimensional hypercube given by . Moreover, let

be a set of polynomials, which we consider as constraints with for all as follows. We define

as the -dimensional hypercube constrained by polynomial inequalities given by .

Throughout the paper we assume that , i.e. the size of the constraint set is polynomial in . This is usually the case, since otherwise the problem gets less tractable from the optimization point of view.

As a first step, we introduce a Kronecker function:

Definition 4.1.

For every the function

(4.1)

is called the Kronecker delta (function) of the vector .

Next we show that the term “Kronecker delta” is justified, i.e., we show that for every the function takes the value zero for all except for where it takes the value one.

Lemma 4.2.

For every it holds that:

Proof.

On the one hand, if , then there exists an index such that . This implies that there exists at least one multiplicative factor in which attains the value zero due to (4.1). On the other hand if then we have

The main result of this section is the following theorem.

Theorem 4.3.

Let . Then   for every if and only if has the following representation:

(4.2)

where , and .

Since we are interested in optimization on the boolean hypercube , we assume without loss of generality that the polynomial considered in Theorem 4.3 has degree at most . Indeed, it has degree bigger than , one can efficiently reduce the degree of by applying iteratively the polynomial division with respect to polynomials with . The remainder of the division process is a polynomial with degree at most that agrees with on all the vertices of .

We begin with proving the easy direction of the equivalence stated in Theorem 4.3.

Lemma 4.4.

If admits a decomposition of the form (4.2), then is nonnegative for all .

Proof.

The coefficients are nonnegative, all are SONC and hence nonnegative on . We have for all , and for all choices of we have for all , and for all . Thus, the right hand side of (4.2) is a sum of positive terms for all . ∎

We postpone the rest of the proof of Theorem 4.3 to the end of the section. Now, we state an result about the presentation of the Kronecker delta function .

Lemma 4.5.

For every the Kronecker delta function can be written as

for and every given as in (2.4) with and given by the hypercube constraints and .

Proof.

First note that the function can be rewritten as

where . Now, the proof follows just by noting that for every both inequalities and are in . ∎

The following statement is well-known in similar variations; see e.g. [BS14, Lemma 2.2 and its proof]. For clarity, we provide an own proof here.

Proposition 4.6.

Let