Constant-Depth and Subcubic-Size Threshold Circuits for Matrix Multiplication

06/25/2020
by   Ojas Parekh, et al.
0

Boolean circuits of McCulloch-Pitts threshold gates are a classic model of neural computation studied heavily in the late 20th century as a model of general computation. Recent advances in large-scale neural computing hardware has made their practical implementation a near-term possibility. We describe a theoretical approach for multiplying two N by N matrices that integrates threshold gate logic with conventional fast matrix multiplication algorithms, that perform O(N^ω) arithmetic operations for a positive constant ω < 3. Our approach converts such a fast matrix multiplication algorithm into a constant-depth threshold circuit with approximately O(N^ω) gates. Prior to our work, it was not known whether the Θ(N^3)-gate barrier for matrix multiplication was surmountable by constant-depth threshold circuits. Dense matrix multiplication is a core operation in convolutional neural network training. Performing this work on a neural architecture instead of off-loading it to a GPU may be an appealing option.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/04/2023

Matrix Multiplication Using Only Addition

Matrix multiplication consumes a large fraction of the time taken in man...
research
09/27/2021

Improved algorithms for Boolean matrix multiplication via opportunistic matrix multiplication

Karppa Kaski (2019) proposed a novel type of "broken" or "opportunis...
research
05/09/2019

An Algebraic Approach to Fast Estimation of the Threshold Voltage of Junctionless Double Gate MOSFETs Using the Gram Schmidt Method

The effect of decreasing Drain-Induced Barrier Lowering (DIBL) is one of...
research
02/19/2020

Fast Multi-Subset Transform and Weighted Sums Over Acyclic Digraphs

The zeta and Moebius transforms over the subset lattice of n elements an...
research
07/12/2022

Look-ups are not (yet) all you need for deep learning inference

Fast approximations to matrix multiplication have the potential to drama...
research
02/22/2022

Distilled Neural Networks for Efficient Learning to Rank

Recent studies in Learning to Rank have shown the possibility to effecti...
research
06/27/2020

JAMPI: efficient matrix multiplication in Spark using Barrier Execution Mode

The new barrier mode in Apache Spark allows embedding distributed deep l...

Please sign up or login with your details

Forgot password? Click here to reset