We say that a boolean function is a Polynomial Threshold Function of degree if it can be expressed as the sign of a polynomial of degree at most evaluated on the boolean hypercube. For brevity, we will use the term -PTF (or simply PTF, when and are either implicit or irrelevant) to refer to a polynomial threshold function of degree on variables. We say that the coefficients of are the realizing weights of . Note that these realizing weights are not unique, as any sufficiently small perturbation of will not affect its sign on the discrete set . This definition alone is not terribly exciting without restrictions on , as every boolean function on variables can be written as the sign of (and in fact can be written exactly as) a multilinear polynomial of degree . We are interested particularly in the case where is small.
In an influential paper, Craig Gotsman and Nathan Linial [GL94] applied Fourier analytic techniques to the study of PTFs. They were mainly interested in connecting different measures of the complexity of boolean functions, and of low-degree PTFs in particular. One such measure was the Average Sensitivity of a boolean function, defined in Fourier analytic terms. For simplicity, in this paper we use the following (equivalent) combinatorial definition:
For a function , we define its Dichromatic Count to be the number of (unordered) pairs of Hamming neighbors such that .
We say that such a pair of Hamming neighbors is a dichromatic edge of .
The Average Sensitivity of a boolean function is .
Among other things, Gotsman and Linial proved a tight upper bound on the average sensitivity of -PTFs, achieved by the MAJORITY function on variables. They conjectured that this bound generalizes to higher degree PTFs, in that the -PTF of maximal average sensitivity is the obvious symmetric candidate, which alternates signs on the values of closest to .
Conjecture 1.1 (Gotsman-Linial)
Let be the monic univariate polynomial of degree with (non-repeated) roots at the integers closest to of opposite parity from . Let . Then for every -PTF , .
This conjecture was listed as a prominent open problem in [OD14] and [FHHMOSWW14]. If true, it would have many applications in complexity and learning (see for example [HKM09, GS10, Kan12, KW16, CSS16]), although most of the applications would already be implied by an asymptotic version of the conjecture, stated below. Gotsman and Linial proved their conjecture for the case where , and it is also known to be true in the case where . However, it was left open whether the conjecture holds for any . Two weaker versions of this conjecture have since been formulated and studied.
Conjecture 1.2 (Gotsman-Linial - Asymptotic)
Let be an -PTF. Then the average sensitivity .
Conjecture 1.3 (Gotsman-Linial - Weak)
Let be an -PTF. Then the average sensitivity for some function depending only on .
In this paper, we resolve the Gotsman-Linial Conjecture (Conjecture 1.1) for all pairs except the case when is even and . The main result of this paper is the following.
In addition, the conjecture holds in many of the remaining cases.
For all pairs of natural numbers satisfying one of the following criteria, has the greatest average sensitivity among -PTFs.
Our results (and the remaining open cases) are summarized in Figure 1. Although we refute the Gotsman-Linial Conjecture for most cases that are of interest for applications, the asymptotic conjecture (Conjecture 1.2), which would suffice for most known applications, remains open.
were verified with the help of a computer search and a linear program solver (see AppendixA).
The remainder of this paper is structured as follows. We first present some high level intuition relating to the Gotsman-Linial Conjecture. Section 2 contains background information. Section 3 contains constructions of the refutations indicated in Figure 1. Section 4 concludes the paper and presents a revised conjecture.
We start with some very high level intuition as to why the Gotsman-Linial Conjecture might be (approximately) true. The conjecture holds in the case of symmetric PTFs (boolean functions which can be expressed as the sign of a univariate polynomial in the sum of the input bits). This follows from the Fundamental Theorem of Algebra and a simple counting argument. In the more general case, we might expect that a degree- PTF can be expressed (at least approximately) in terms of unate functions. This generalizes the observation that every linear threshold function is unate. For a sufficiently close approximation, this would prove the Asymptotic Gotsman-Linial Conjecture. Intuition may also be drawn from Kane’s proof of Conjecture 1.3is expected to be too large in magnitude for a small change in its input to change its sign. Under certain conditions, a similar result can be extended to polynomial threshold functions on the boolean hypercube.
As for why the Gotsman-Linial Conjecture is not (exactly) true, we observe that the PTF of conjectured maximal average sensitivity is the product of
linear threshold functions, with parallel separating hyperplanes between two of the middlelayers (sets of vertices of equal Hamming weight) in the hypercube. For some , one might expect to be able to find a PTF of greater average sensitivity approximated by turning one of these separating hyperplanes ‘sideways’, i.e. replacing a hyperplane that cuts the fewest edges with a hyperplane orthogonal to the rest. Intuitively, this would require that be sufficiently large that some of the hyperplanes cut many more edges than others, but also sufficiently small that not too many edges are cut by two hyperplanes. As it turns out, this intuition can be formalized for many and , refuting the Gotsman-Linial Conjecture.
Low-degree PTFs, in particular linear threshold functions (degree- PTFs) with integral and polynomially bounded realizing weights, are of interest in the study of complexity classes such as
(i.e. circuits composed of AND, OR, NOT, and MAJORITY gates of unbounded fan-in) and of neural networks. More generally, we say that a circuit (with unbounded fan-in) is adegree- polynomial threshold circuit if each of its constituent gates computes a degree- PTF of its inputs. Note that since AND, OR, MAJORITY, and NOT are all linear threshold functions, and circuits are degree- polynomial threshold circuits. Despite much research, the power of polynomial threshold circuits is poorly understood. For instance, it is currently an open question, and a rather embarrassing one at that, whether (the class of functions computable in nondeterministic time) is contained in (the class of functions computable by families of depth-, polynomial size linear threshold circuits with polynomially bounded realizing weights). Recent work by Daniel Kane and Ryan Williams [KW16] gave a partial answer to this question. They studied the sensitivity of PTFs to random restrictions, proving (among other things) that (and in fact, -uniform ) does not have depth-3 circuits of gates or wires.
Conjecture 1.1 is trivially true in the cases and (the only -PTFs are the constant functions, and is the parity function, which has the maximum possible average sensitivity). Gotsman and Linial originally noted that Conjecture 1.1 had already been proven in the case where by Patrick O’Neil in 1971 [ON71].
Theorem 2.1 (O’Neil)
The maximal number of edges of which may be cut by a hyperplane is given by .
Very little additional progress was made towards resolving the above conjectures until recently. The first non-trivial bounds on the average sensitivity of PTFs of arbitrary degree were found independently by two groups [HKM09, DRST14] and published jointly [DHKMRST10]. Daniel Kane in 2012 obtained the first bound which was truly sublinear in [Kan12], and in 2013, he proved the weak version of the Gotsman-Linial Conjecture (Conjecture 1.3) [Kan13].
3 Resolution of Gotsman-Linial Conjecture
For simplicity, we start by introducing some notation.
Let . We say , iff there exist and such that the function is a constant.
Note that defines an equivalence relation on boolean functions. Two functions are equivalent iff one can be turned into the other through a combination of permuting the inputs and negating the inputs/output.
An -Hypersensitive Function, or -HSF is an -PTF such that .
More generally, we say that a PTF is an HSF if and are either implicit or irrelevant. We may now restate the original Gotsman-Linial Conjecture (Conjecture 1.1) as follows:
For all , -HSFs do not exist.
For every , -HSFs do not exist.
Proof. Every -PTF is defined by a separating hyperplane which cuts all of the dichromatic edges of . From O’Neil, , so is not an HSF.
The case is a simple consequence of a result first proven in 1968 by Marvin Minsky and Seymour Papert [MP68] and since re-proven several times. We present here a variation on the proof by Aspnes et al. [ABFR94].
Theorem 3.1 (Minsky-Papert)
Any PTF which computes parity on variables must have degree at least .
Proof. Let be a multilinear polynomial of degree which is never zero on . The set of monomials of degree at most
is an orthogonal basis for the vector space of degree-multilinear polynomials on the boolean hypercube. Hence is orthogonal to the parity function , i.e. . By assumption, every term in the sum on the RHS is non-zero, so at least one of them is negative, i.e. .
For every , -HSFs do not exist.
Proof. Let be an -PTF. Then , and . Let and . Take and . There are edge-disjoint paths between and in the boolean hypercube, and each must contain at least one edge crossing the cut between and (i.e. a monochromatic edge). Hence , so is not an HSF.
Let , and let have maximal over all -PTFs. Then for every -PTF , .
Proof. Let , and let be an -PTF with maximal. Let be an -PTF. Any restriction of to a function on variables is also a degree- PTF, so . There are such restrictions , and each dichromatic edge of appears in exactly of them. Hence , from which the desired result follows immediately.
Let with . If and have the same parity, and -HSFs do not exist, then -HSFs do not exist.
Proof. Assume that no -PTF is an HSF. If and have the same parity, then every restriction of is equivalent (with respect to ) to . There are such restrictions, and each dichromatic edge of appears in exactly of them, so . Hence by Lemma 3.1, -HSFs do not exist.
For every , -HSFs do not exist.
Let . If and , then -HSFs do not exist.
3.1 A Simple Counterexample
In the statement of Corollary 3.4, the caveat cannot be removed.
There exists a unique -HSF , modulo .
Proof. In the case where and , , and . Let be defined by , let such that , and let . Since is quadratic, is a -PTF. It is not difficult to verify that , so is a -HSF. For uniqueness, see Appendix A.1.
The existence of a -HSF precludes the use of Lemma 3.2 to prove that -HSFs do not exist. However, the uniqueness of , along with the fact that it only has one additional dichromatic edge, allows for a proof using Lemma 3.1.
For every , -HSFs do not exist.
Proof. The cases have already been covered. For , see Appendix A.3. The case remains. Assume for the sake of contradiction that is a -HSF. The dichromatic count of every boolean function on an even number of variables is an even integer. Since for every -PTF , , Lemma 3.1 implies that , and hence that . There are restrictions of to a function on variables, all of which satisfy . Every dichromatic edge in appears in exactly five such , so the expectation over a uniformly random restriction of is . Since is always an integer,
with probability strictly greater than. In particular, there exists such that (*). However, it is easily verified (see Appendix A.2) that no function satisfying both (*) and is a -PTF. This contradicts the initial choice of . Hence no -HSFs exist.
This also completes the proof of Theorem 1.2.
3.2 Extension to Odd
We may extend to an -HSF for any odd .
For every odd with , there exists an -HSF with
Intuitively, behaves exactly as , with the additional variables contributing to the second argument of .
Proof. Let be an odd integer. Let and . Let be the -dimensional boolean hypercube, and let be the graph with vertex set and an edge between and exactly when . Let be the graph homomorphism defined by . Let as above, let , and take . Note that because is a graph homomorphism, we may compute by counting the dichromatic edges induced by on , weighted by . To this end, we observe that an edge between and has a preimage under of cardinality
Similarly, for an edge between and ,
We observe that is positive on except at the four points . Hence gives nine dichromatic edges, as indicated by the black lines below.
Summing the above expressions over these nine edges, we have
Hence is an -HSF, as desired.
3.3 The General Case
Using a similar construction, we now prove the existence of HSFs of arbitrary degree.
For every with and , there exists an -HSF with .
We first consider the case where and have the same parity. The case where and have opposite parity is similar but handled later.
For every with and even, there exists an -HSF with .
Proof. Let be integers of the same parity with . Let and let . Let be the -dimensional boolean hypercube, and let be the graph with vertex set and an edge between and exactly when . Let be the graph homomorphism defined by . We now define four polynomials on as follows:
Since , there exists such that for every with , . Similarly, , so there exists such that for every with , . For instance, we may take . Take , take , and take . Since and have degree , has degree , and has degree , is a polynomial threshold function of degree . Towards computing , we first consider the relevant behaviors of separately. All four are integer-valued (evaluations of) polynomials on the domain . Both and are always odd, so in particular, are non-zero everywhere. Firstly, is positive when , is zero when , and has the same sign as when . Clearly, is positive when and zero when . By choice of , is never in the interval and always has the same sign as when is non-zero. Similarly, is always non-zero and always has the same sign as when is non-zero. Hence we may rewrite as the following piecewise function:
We now define by . Note that because is symmetric, this gives a well-defined function . It is easily verified that for all such that and at the two points and , , and that for all other , . Hence there are ten edges in for which . This allows us to compute as follows:
Hence is an -HSF, as desired.
For every with , and odd, there exists an -HSF with .
Proof. The proof proceeds similarly to the previous case. We define , , , , , , , , , , and as above, and we define . We now define analogously to above. The computation of now proceeds as follows:
Hence is an -HSF, as desired.
For almost all and almost all , we refute the Gotsman-Linial Conjecture (Conjecture 1.1) with a multiplicative separation of . This separation is too weak to refute most known applications of the conjecture. We would need to improve to to refute the Asymptotic Gotsman-Linial Conjecture (Conjecture 1.2), on which the applications depend. Although for every -HSF given in this paper, , it should be noted that the RHS is still an upper bound in a limiting sense. This, along with the intuition presented in Section 1.2, invites the following revised conjecture.
Conjecture 4.1 (Gotsman-Linial - Limit)
Let be an -PTF. Then the average sensitivity .
For every even , -HSFs do not exist.
Furthermore, our revised conjecture would imply the Asymptotic Gotsman-Linial Conjecture (Conjecture 1.2) and its consequent applications.
The author would like to thank Ryan Williams especially for inspiration, advice, feedback, and an admirable tolerance of cheesemonkeys; the Williams family, the Chap-people, Henry Qin, and Carolyn Kim for moral support and a good work environment; and Not Luke the goldfish for surviving.
- [ABFR94] James Aspnes, Richard Biegel, Merrick Furst, and Steven Rudich. The Expressive Power of Voting Polynomials. Combinatorica, 14(2):1-14, 1994.
- [Bou79] Jean Bourgain. Walsh subspaces of product spaces. Séminaire D’Analyse Fonctionelle, École Polytechnique, Centre de Mathematiques, pp. IV.1-IV.9, 1979.
- [CSS16] Ruiwen Chen, Rahul Santhanam, and Srikanth Srinivasan. Average-Case Lower Bounds and Satisfiability Algorithms for Small Threshold Circuits. Proceedings for the st Conference on Computational Complexity, pp. 1:1-1:35, 2016.
Ilias Diakonikolas, Prahladh Harsha, Adam Klivans, Raghu Meka, Prasad Raghavendra, Rocco A. Servedio, and Li-Yang Tan.
Bounding the average sensitivity and noise sensitivity of polynomial threshold functions.
ACM Symposium on Theory of Computing (STOC), pp. 533–542, 2010.
- [DRST14] Ilias Diakonikolas, Prasad Raghavendra, Rocco A. Servedio, and Li-Yang Tan. Average sensitivity and noise sensitivity of polynomial threshold functions. SIAM Journal on Computing, pp. 231–253, 2014.
- [FHHMOSWW14] Yuval Filmus, Hamed Hatami, Steven Heilman, Elchanan Mossel, Ryan O’Donnell, Sushant Sachdeva, Andrew Wan, and Karl Wimmer. Real Analysis in Computer Science: A collection of Open Problems. Simons Institute, Berkeley, CA, compiled in 2014. URL: https://simons.berkeley.edu/sites/default/files/openprobsmerged.pdf
- [GS10] Parikshit Gopalan and Rocco Servedio. Learning and Lower Bounds for with Threshold Gates. Proceedings for the th International Workshop on Randomization and Computation, pp. 588-601, 2010.
- [GL94] Craig Gotsman and Nathan Linial. Spectral Properties of Threshold Functions. Combinatorica, 14(1):35-50, 1994.
- [HKM09] Prahladh Harsha, Adam Klivans and Raghu Meka. Bounding the sensitivity of polynomial threshold functions. arXiv:0909.5175, 2009.
- [Kan12] Daniel M. Kane. A structure theorem for poorly anticoncentrated Gaussian chaoses and applications to the study of polynomial threshold functions. In Foundations of Computer Science (FOCS), pp. 91–100, 2012.
- [Kan13] Daniel M. Kane. The Correct Exponent for the Gotsman-Linial Conjecture. arXiv:1210.1283, 2013.
- [KW16] Daniel Kane and Ryan Williams. Super-Linear Gate and Super-Quadratic Wire Lower Bounds for Depth-Two and Depth-Three Threshold Circuits. Proceedings for the th Annual ACM SIGACT Symposium on the Theory of Computing, pp. 633-643, 2016.
Nathan Linial, Yishay Mansour, and Noam Nisan.
Constant depth circuits, Fourier transform, and learnability.Journal of the ACM, 40(3):607-620, 1993.
- [MP68] Marvin Minsky and Seymour Papert. Perceptrons: an Introduction to Computational Geometry (Expanded Edition). MIT Press, Cambridge, MA, 1988.
- [OD12] Ryan O’Donnell. Open problems in analysis of Boolean functions. arXiv preprint arXiv:1204.6447, 2012.
- [OD14] Ryan O’Donnell. Analysis of Boolean Functions. Cambridge University Press, New York, New York, 2014.
- [ON71] Patrick E. O’Neil. Hyperplane Cuts of an -Cube. Discrete Maths. 1 (1971), 193-195.
Appendix A Appendix
Here we describe how a computer search resolved the cases of and of the Gotsman-Linial Conjecture. First, we note that the problem of determining whether a boolean function is an -PTF is equivalent to determining whether a particular linear program has any feasible solution. Unfortunately, leveraging this fact to compute the maximal average sensitivity of an -PTF with a naïve exhaustive search takes doubly exponential time so is intractable for large (i.e. ). However, by using Lemma 3.1, we can conduct a more efficient search. We maintain a partial function and conduct a DFS in which we define successively on inputs in increasing order of Hamming weight. This allows us to keep bounds on by counting the edges that are already constrained to be monochromatic or dichromatic. When becomes too low or too high, we can prune the search and backtrack before fully defining , allowing (tolerably) efficient searches up to .
In the case , Lemma 3.1 implies that for every -HSF , we have . Because a random boolean function on variables has far fewer than dichromatic edges with high probability, most search branches are pruned early. The modified search confirmed that every -HSF with satisfies , and hence that is the unique -HSF.
The proof of Lemma 3.4 relies on the claim that for every -PTF