On top fan-in vs formal degree for depth-3 arithmetic circuits

04/10/2018
by   Mrinal Kumar, et al.
0

We show that over the field of complex numbers, every homogeneous polynomial of degree d can be approximated (in the border complexity sense) by a depth-3 arithmetic circuit of top fan-in at most d+1. This is quite surprising since there exist homogeneous polynomials P on n variables of degree 2, such that any depth-3 arithmetic circuit computing P must have top fan-in at least Ω(n). As an application, we get a new tradeoff between the top fan-in and formal degree in an approximate analog of the celebrated depth reduction result of Gupta, Kamath, Kayal and Saptharishi [GKKS13]. Formally, we show that if a degree d homogeneous polynomial P can be computed by an arithmetic circuit of size s, then for every t ≤ d, P is in the border of a depth-3 circuit of top fan-in s^O(t) and formal degree s^O(d/t). To the best of our knowledge, the upper bound on the top fan-in in the original proof of [GKKS13] is always at least s^O(√(d)), regardless of the formal degree.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/15/2018

Arithmetic Circuits with Locally Low Algebraic Rank

In recent years, there has been a flurry of activity towards proving low...
03/06/2019

Closure of VP under taking factors: a short and simple proof

In this note, we give a short, simple and almost completely self contain...
05/29/2021

Verified Quadratic Virtual Substitution for Real Arithmetic

This paper presents a formally verified quantifier elimination (QE) algo...
10/24/2017

On the Polynomial Parity Argument Complexity of the Combinatorial Nullstellensatz

The complexity class PPA consists of NP-search problems which are reduci...
04/15/2020

Learning sums of powers of low-degree polynomials in the non-degenerate case

We develop algorithms for writing a polynomial as sums of powers of low ...
03/12/2021

Efficient reconstruction of depth three circuits with top fan-in two

We develop efficient randomized algorithms to solve the black-box recons...
12/09/2019

Approximating the Determinant of Well-Conditioned Matrices by Shallow Circuits

The determinant can be computed by classical circuits of depth O(log^2 n...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

An arithmetic circuit on variables over a field is a directed acyclic graph with leaves labeled by variables in and constants in and internal vertices labeled by sum or product . Such a circuit provides a natural succinct representation for multivariate polynomials in . In this paper, the principle object of interest will be arithmetic circuits of depth-, which we now define.

A depth- circuit (denoted by ) is an arithmetic circuit whose internal gates are arranged in three layers of alternating sums and products with the top layer being a sum. Such a circuit gives a representation of a polynomial as a sum of products of affine forms. Two of the parameters of interest of a circuit are the fan-in of the top layer, which we call the top fan-in of and the maximum of the degrees of its product gates, which we call the formal degree of .

A crucial point to note is that the formal degree of a circuit can be much much larger than the degree of the polynomial computed by . Such a circuit computes polynomials of really high degree by taking products of affine forms, and then takes a linear combination of many such high degree polynomials to efficiently compute a much lower degree polynomial via cancellations. One classic example of this is a result of Ben-Or [NW97] who showed that over large enough fields, for every degree , the elementary symmetric polynomial of degree in variables defined as

can be computed by a circuit with top fan-in and formal degree . In sharp contrast, Nisan and Wigderson [NW97] had earlier shown that any circuit computing of formal degree at most must have top fan-in at least (for a very large range of choice of ). Thus, higher formal degree indeed helps make the computation efficient.

Another recent example of the power non-homogeneous depth- circuits is a beautiful result of Gupta, Kamath, Kayal and Saptharishi [GKKS13], who showed that over the field , there is a circuit of formal degree and top fan-in which computes the determinant of an symbolic matrix. Prior to this work, the best circuit known for the determinant had size . In fact, in [GKKS13], the authors prove something stronger. They show that over the field , if a homogeneous -variate polynomial of degree can be computed by an arithmetic circuit of size , then it can be computed by an circuit of top fan-in and formal degree .

Thus, increasing the the formal degree of circuits helps reduce their top fan-in (and size) while preserving the expressiveness. The main motivation for this work is to explore this tradeoff between the formal degree and top fan-in better. In particular, the following question is of interest to us.

Question 1.1.

In the depth reduction results of [GKKS13], can we increase the formal degree of the resulting circuit further and obtain an circuit with a smaller top fan-in ?

To the best of our understanding, the upper bound on the top fan-in of the circuit obtained by depth reducing a general circuit of size computing a degree polynomial is , and the formal degree is . Here, . Thus, regardless of the choice of and the resulting formal degree obtained, the top fan-in upper bound is always . Before we state our results, we make a brief tour into the realm of approximative or border algebraic computation, since the notion is crucial to our results here.

1.1 Approximative or Border Complexity

Let be a measure of complexity of multivariate polynomials with respect to any reasonable model of computation. For instance, we can think of to be circuit complexity, formula complexity, or depth- circuit complexity of . We say that has border complexity with respect to at most (denoted as ) iff there are polynomials such that the polynomial satisfies . For brevity, we say that the polynomial approximates the polynomial . This notion of approximation is often referred to as the algebraic approximation, as opposed to the usual topological notion. For this paper, we only work with algebraic approximation upper bounds, which trivially imply upper bounds in the topological sense as well. We refer the reader to an excellent discussion on border complexity in the work of Bringmann et al. [BIZ17].

It follows from the definitions that trivially implies , but in general we do not know implications in the other direction. In particular, it is potentially easier to prove border complexity upper bounds for a model, and harder to prove border complexity lower bounds. And, indeed we know some upper bounds in the border complexity framework which are either false, or not known in the exact computation framework. We state two such results.

Low degree factors of polynomials with small circuits.

In [Bür04], Bürgisser showed that if a polynomial has circuit complexity at most , and is an irreducible factor of of degree , then has border circuit complexity at most . In particular, this upper bound does not depend on the degree of itself, which could be as large as ! For exact computation, we only know that can be computed by an arithmetic circuit of size , where is the largest integer such that divides . Note that can be as large as , and hence the bound is not always polynomially bounded in and . Extending the results in [Bür04] continues to be a fascinating and fundamental open problem.

Width- algebraic branching programs.

In [BIZ17], Bringmann, Ikenmeyer and Zuiddam showed that over all fields of characteristic different from , if a polynomial of degree has an arithmetic formula of size , then is in the border of width two algebraic branching programs of size at most . This is the approximative version of a strengthening of a classical result of Ben-Or and Cleve [BC88] who showed that if a degree polynomial has an arithmetic formula of size , then can be computed by a width- algebraic branching program of size . The result in [BIZ17] is surprising, since we know that an analog of the result of Ben-Or and Cleve is false for width- algebraic branching programs. In fact, Allender and Wang [AW16] showed that width- algebraic branching programs are not even complete, i.e. there are polynomials which they cannot compute regardless of the size.

1.2 Results

Our main result is the following theorem.

Theorem 1.2 (Approximating general polynomials).

Let be any homogeneous polynomial of degree . Then, there exists a circuit with top fan-in at most and formal degree at most such that

where and every monomial with a non-zero coefficient in has degree strictly greater than .

Thus, every homogeneous polynomial of degree is in the border of circuits with top fan-in at most . Of course, the upper bound on the formal degree is extremely high, and up to lower order terms, this is unavoidable due to counting arguments. We remark that this result is a bit surprising since it known to be false in the realm of exact computation. More formally, the following folklore lemma is well known (at least implicitly).

Lemma 1.3 (Follows from Lemma 4.9 in [Sw01], Lemma A.1 in [Cgj16]).

Any circuit computing the inner product polynomial must have top fan-in , regardless of the formal degree.

Thus, the exact computation analog of Theorem 1.2 is false in a very strong sense, even for polynomials of degree .

An interesting implication of Theorem 1.2 is that one cannot hope to prove super linear top fan-in lower bound for circuits for polynomials of degree without taking into account the formal degree, provided the lower bound also applies to the border of circuits. Most of the known lower bound results for arithmetic circuits do in fact extend to the border of the corresponding complexity class.

Our second theorem is a special case of Theorem 1.4 for sums of powers of linear forms. The upper bound on the formal degree of the approximating circuit obtained here is much better.

Theorem 1.4 (Approximating sums of powers of linear forms).

Let be any homogeneous polynomial of degree in , where each is a homogeneous linear form. Then, there exists a circuit with top fan-in at most and formal degree at most such that

where and every monomial with a non-zero coefficient in has degree strictly greater than .

Our final result answers 1.1 in the affirmative in the border complexity sense. We prove the following statement.

Theorem 1.5 (Top fan-in vs formal degree for chasm at depth-).

Let be a homogeneous polynomial of degree which is computable by an arithmetic circuit of size . Then, for every , there is a circuit of top fan-in at most and formal degree such that

where and every monomial with a non-zero coefficient in has degree strictly greater than .

As remarked earlier, this tradeoff is in contrast to the original result of Gupta, Kamath, Kayal and Saptharishi [GKKS13] where the top fan-in is always at least regardless of the formal degree of the circuit.

In the rest of this note, we include the proofs of the above theorems. All our proofs are based on very simple and elementary ideas building on top of known results in this area, most notably those in [Shp02, GKKS13]. However, the theorem statements, and in particular Theorem 1.2 seems to be interesting (and surprising!).

2 Preliminaries

  • Unless otherwise stated, we work over the field of complex numbers.

  • is the number of variables and is the degree.

  • We use boldface letters like to denote the set of variables .

  • For a natural number , denotes the set .

  • For a scalar , and a polynomial , .

Theorem 2.1 (Shpilka, Theorem 2.1 in [Shp02]).

Let be any homogeneous polynomial of degree . Then, there exist homogeneous linear forms for , such that

Theorem 2.2 (Shpilka, Lemma 2.4 in [Shp02]).

Let be any homogeneous polynomial of degree in , where each is a homogeneous linear form. Let be a primitive root of unity of order . Then,

3 Proofs and technical details

Lemma 3.1 (Approximating low degree homogeneous components).

Let be any field of size at least and let be a polynomial of degree in where for each , is the homogeneous component of of degree equal to . Then, for every and for every choice of distinct elements in , there exist in such that

where the degree of every monomial with a non-zero coefficient in is at least .

Proof.

Let be a new formal variable, and let be the defined as

Clearly, . For the rest of the proof, we fix an arbitrary . Let be any distinct elements of . Then, for every , we have

For , let . From the choice of , we know that for any , . Thus, it follows that are linearly independent, and hence there exist scalars in such that

(3.2)

Therefore,

From Equation 3.2, we know that for every ,

and,

Thus,

Lemma 3.3 (Approximating powers of diagonal circuits).

Let be homogeneous linear forms and let be a homogeneous polynomial of degree for any . Then, there exists a circuit of top fan-in at most and formal degree at most such that

where and every monomial with a non-zero coefficient in has degree at least .

Proof.

The proof follows the proof of the original duality lemma of Saxena [Sax08]

except in place of the interpolation used there (which blows up the top fan-in by a factor as large as

), we only do a partial interpolation in the spirit of 3.1 here, and this only incurs a small blow up in the top fan-in. We now sketch some of the details.

By , we denote the formal power series , and by we denote the truncation of this power series to monomials of degree at most , i.e. . Now, observe that

(3.4)
(3.5)

We are now in a scenario similar to what we handled in the proof of 3.1; we have a polynomial which we think of as univariate in , and we are interested in the coefficient of in this polynomial which has degree . From the proof of 3.1, we know that for any distinct field elements , there exist field elements such that

where is some linear combination of monomials of degree at least . Also, note that for every , can be written as a product of linear functions. Thus, can be computed by a circuit with top fan-in and formal degree . Replacing every in by and dividing by completes the proof of the lemma. ∎

Approximating general homogeneous polynomials.

Proof of Theorem 1.2.

Let be any homogeneous degree polynomial. Then, from Theorem 2.1, we know that for there are homogeneous linear forms such that

It was observed by Michael Ben-Or (see [NW97])111and, is easy to see that equals the homogeneous component of of degree equal to . Thus, from 3.1, we know that there exist scalars and such that

Replacing every in by and dividing by completes the proof of the theorem.

Approximating sums of powers of linear forms.

Proof of Theorem 1.4.

The proof of Theorem 1.4 is essentially the same as the proof of Theorem 1.2, except for the fact that we use Theorem 2.2 in the place of Theorem 2.1, and this gives us the desired upper bound on formal degree of in terms of its Waring rank, as opposed to sparsity. ∎

Approximate reduction to depth-.

We present two proofs of Theorem 1.5, both simple, but one simpler than the other.

First proof of Theorem 1.5.

Let be a parameter. As a first step, we use the standard depth reduction to depth- [AV08, Tav15] to transform a homogeneous circuit of unbounded depth to a homogeneous circuit , with and . Now, for every sub-circuit at the bottom two levels, we apply Theorem 1.2, which says that each of these sub-circuits can be approximated by circuits. We now plug these depth- circuits into to get a circuit . Expanding out the product gates at the second level by brute force, we obtain a circuit which approximates the polynomial computed by . Combining the sum gates in the first two layers gives us the desired depth- circuit. It is also not hard to see that the polynomial computed by the resulting depth- circuit approximates the polynomial computed by in the sense of the statement of Theorem 1.5. We skip the details. ∎

Second proof of Theorem 1.5.

For this proof, we will follow the outline in [GKKS13], with one difference. In place of the use of the duality lemma of Saxena to transform a homogeneous depth- powering circuit to a non-homogeneous depth- circuit, we use 3.3. We now elaborate on some of the details, and chase the parameters in the process. We refer the reader to the original paper of Gupta et al [GKKS13] for details.

Let be a parameter. We first transform a homogeneous circuit of unbounded depth to a homogeneous circuit using the standard depth reduction to depth- [AV08, Tav15] and Fischer’s formula (Lemma IV.3 in [GKKS13]). It follows from the proofs of Lemma IV.4 and Lemma IV.2 in [GKKS13] that and .

Now, we apply 3.3 to each sub-circuit, and take their sum. This gives us a circuit with top fan-in and formal degree . This completes the proof. ∎

4 Open problems

We end with some open problems.

  • Perhaps the most natural question is to understand if the exact computation versions of Theorem 1.2, Theorem 1.4 or Theorem 1.5 are true with a reasonable blow up in the parameters. For instance, can every homogeneous polynomial of degree be computed by a circuit with top fan-in , is arbitrarily large formal degree is allowed ? What about all polynomials in ?

  • Another question is to understand if there are natural classes of polynomials for which the formal degree upper bound in Theorem 1.2 can be reduced to some more reasonable (and possibly useful) upper bound, while keeping the top fan-in small. For instance, can every homogeneous degree polynomial with a formula of size be approximated by a circuit of top fan-in and formal degree ?

  • And finally, Theorem 1.2 seems to be a very general structural statement for low degree polynomials. Does this have other applications ?

Acknowledgements.

I am thankful to Michael Forbes, Amir Shpilka and Ramprasad Saptharishi for helpful discussions.

References