DeepAI

Kronecker Products, Low-Depth Circuits, and Matrix Rigidity

For a matrix M and a positive integer r, the rank r rigidity of M is the smallest number of entries of M which one must change to make its rank at most r. There are many known applications of rigidity lower bounds to a variety of areas in complexity theory, but fewer known applications of rigidity upper bounds. In this paper, we use rigidity upper bounds to prove new upper bounds in a few different models of computation. Our results include: ∙ For any d> 1, and over any field 𝔽, the N × N Walsh-Hadamard transform has a depth-d linear circuit of size O(d · N^1 + 0.96/d). This circumvents a known lower bound of Ω(d · N^1 + 1/d) for circuits with bounded coefficients over ℂ by Pudlák (2000), by using coefficients of magnitude polynomial in N. Our construction also generalizes to linear transformations given by a Kronecker power of any fixed 2 × 2 matrix. ∙ The N × N Walsh-Hadamard transform has a linear circuit of size ≤ (1.81 + o(1)) N log_2 N, improving on the bound of ≈ 1.88 N log_2 N which one obtains from the standard fast Walsh-Hadamard transform. ∙ A new rigidity upper bound, showing that the following classes of matrices are not rigid enough to prove circuit lower bounds using Valiant's approach: - for any field 𝔽 and any function f : {0,1}^n →𝔽, the matrix V_f ∈𝔽^2^n × 2^n given by, for any x,y ∈{0,1}^n, V_f[x,y] = f(x ∧ y), and - for any field 𝔽 and any fixed-size matrices M_1, …, M_n ∈𝔽^q × q, the Kronecker product M_1 ⊗ M_2 ⊗⋯⊗ M_n. This generalizes recent results on non-rigidity, using a simpler approach which avoids needing the polynomial method.

11/09/2022

Smaller Low-Depth Circuits for Kronecker Powers

We give new, smaller constructions of constant-depth linear circuits for...
03/09/2021

Improved upper bounds for the rigidity of Kronecker products

The rigidity of a matrix A for target rank r is the minimum number of en...
03/29/2020

A Polynomial Degree Bound on Defining Equations of Non-rigid Matrices and Small Linear Circuits

We show that there is a defining equation of degree at most 𝗉𝗈𝗅𝗒(n) for ...
05/17/2022

On Algebraic Constructions of Neural Networks with Small Weights

Neural gates compute functions based on weighted sums of the input varia...
12/07/2015

Gauss quadrature for matrix inverse forms with applications

We present a framework for accelerating a spectrum of machine learning a...
11/14/2022

Low-depth arithmetic circuit lower bounds via shifted partials

We prove super-polynomial lower bounds for low-depth arithmetic circuits...
04/02/2019

Lower Bounds for Matrix Factorization

We study the problem of constructing explicit families of matrices which...