DeepAI AI Chat
Log In Sign Up

Kronecker Products, Low-Depth Circuits, and Matrix Rigidity

by   Josh Alman, et al.
Harvard University

For a matrix M and a positive integer r, the rank r rigidity of M is the smallest number of entries of M which one must change to make its rank at most r. There are many known applications of rigidity lower bounds to a variety of areas in complexity theory, but fewer known applications of rigidity upper bounds. In this paper, we use rigidity upper bounds to prove new upper bounds in a few different models of computation. Our results include: ∙ For any d> 1, and over any field 𝔽, the N × N Walsh-Hadamard transform has a depth-d linear circuit of size O(d · N^1 + 0.96/d). This circumvents a known lower bound of Ω(d · N^1 + 1/d) for circuits with bounded coefficients over ℂ by Pudlák (2000), by using coefficients of magnitude polynomial in N. Our construction also generalizes to linear transformations given by a Kronecker power of any fixed 2 × 2 matrix. ∙ The N × N Walsh-Hadamard transform has a linear circuit of size ≤ (1.81 + o(1)) N log_2 N, improving on the bound of ≈ 1.88 N log_2 N which one obtains from the standard fast Walsh-Hadamard transform. ∙ A new rigidity upper bound, showing that the following classes of matrices are not rigid enough to prove circuit lower bounds using Valiant's approach: - for any field 𝔽 and any function f : {0,1}^n →𝔽, the matrix V_f ∈𝔽^2^n × 2^n given by, for any x,y ∈{0,1}^n, V_f[x,y] = f(x ∧ y), and - for any field 𝔽 and any fixed-size matrices M_1, …, M_n ∈𝔽^q × q, the Kronecker product M_1 ⊗ M_2 ⊗⋯⊗ M_n. This generalizes recent results on non-rigidity, using a simpler approach which avoids needing the polynomial method.


page 1

page 2

page 3

page 4


Smaller Low-Depth Circuits for Kronecker Powers

We give new, smaller constructions of constant-depth linear circuits for...

Improved upper bounds for the rigidity of Kronecker products

The rigidity of a matrix A for target rank r is the minimum number of en...

A Polynomial Degree Bound on Defining Equations of Non-rigid Matrices and Small Linear Circuits

We show that there is a defining equation of degree at most 𝗉𝗈𝗅𝗒(n) for ...

On Algebraic Constructions of Neural Networks with Small Weights

Neural gates compute functions based on weighted sums of the input varia...

Gauss quadrature for matrix inverse forms with applications

We present a framework for accelerating a spectrum of machine learning a...

Low-depth arithmetic circuit lower bounds via shifted partials

We prove super-polynomial lower bounds for low-depth arithmetic circuits...

Lower Bounds for Matrix Factorization

We study the problem of constructing explicit families of matrices which...