Circuit Complexity of Visual Search

by   Kei Uchizawa, et al.

We study computational hardness of feature and conjunction search through the lens of circuit complexity. Let x = (x_1, ... , x_n) (resp., y = (y_1, ... , y_n)) be Boolean variables each of which takes the value one if and only if a neuron at place i detects a feature (resp., another feature). We then simply formulate the feature and conjunction search as Boolean functions FTR_n(x) = ⋁_i=1^n x_i and CONJ_n(x, y) = ⋁_i=1^n x_i ∧ y_i, respectively. We employ a threshold circuit or a discretized circuit (such as a sigmoid circuit or a ReLU circuit with discretization) as our models of neural networks, and consider the following four computational resources: [i] the number of neurons (size), [ii] the number of levels (depth), [iii] the number of active neurons outputting non-zero values (energy), and [iv] synaptic weight resolution (weight). We first prove that any threshold circuit C of size s, depth d, energy e and weight w satisfies log rk(M_C) ≤ ed (log s + log w + log n), where rk(M_C) is the rank of the communication matrix M_C of a 2n-variable Boolean function that C computes. Since CONJ_n has rank 2^n, we have n ≤ ed (log s + log w + log n). Thus, an exponential lower bound on the size of even sublinear-depth threshold circuits exists if the energy and weight are sufficiently small. Since FTR_n is computable independently of n, our result suggests that computational capacity for the feature and conjunction search are different. We also show that the inequality is tight up to a constant factor if ed = o(n/ log n). We next show that a similar inequality holds for any discretized circuit. Thus, if we regard the number of gates outputting non-zero values as a measure for sparse activity, our results suggest that larger depth helps neural networks to acquire sparse activity.


page 1

page 2

page 3

page 4


New Bounds for Energy Complexity of Boolean Functions

For a Boolean function f:{0,1}^n →{0,1} computed by a circuit C over a...

Functional lower bounds for restricted arithmetic circuits of depth four

Recently, Forbes, Kumar and Saptharishi [CCC, 2016] proved that there ex...

A #SAT Algorithm for Small Constant-Depth Circuits with PTF gates

We show that there is a randomized algorithm that, when given a small co...

Computing the Best Case Energy Complexity of Satisfying Assignments in Monotone Circuits

Measures of circuit complexity are usually analyzed to ensure the comput...

Parallel RAM from Cyclic Circuits

Known simulations of random access machines (RAMs) or parallel RAMs (PRA...

Criticality of AC^0 formulae

Rossman [In Proc. 34th Comput. Complexity Conf., 2019] introduced the no...

SAT-based Circuit Local Improvement

Finding exact circuit size is a notorious optimization problem in practi...

Please sign up or login with your details

Forgot password? Click here to reset