A Logical Characterization of Constant-Depth Circuits over the Reals

05/11/2020 ∙ by Timon Barlag, et al. ∙ uni hannover 0

In this paper we give an Immerman's Theorem for real-valued computation. We define circuits operating over real numbers and show that families of such circuits of polynomial size and constant depth decide exactly those sets of vectors of reals that can be defined in first-order logic on R-structures in the sense of Cucker and Meer. Our characterization holds both non-uniformily as well as for many natural uniformity conditions.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Computational complexity theory is a branch of theoretical computer science which focuses on the study and classification of problems with regard to their innate difficulty. This is done by dividing these problems into classes, according to the amount of resources necessary to solve them using particular models of computation. One of the most prominent such models is the Turing machine – a machine operating sequentially on a fixed, finite alphabet.

If one wishes to study problems based on their parallel complexity or in the domain of the real numbers, one requires different models of computation. Theoretical models exist both for real-valued sequential and for real-valued parallel computation, going back to the seminal work by Blum, Shub and Smale [1]. Unlike Turing machines, machines over obtain not an unstructured sequence of bits as input but a vector of real numbers or an (encoding of an) -structure. The respective parallel model we are going to have a closer look at is a real analogue to the arithmetic circuit (see, e.g., [12]), which, as its name suggests, resembles electrical circuits in its functioning, however, contrary to these they operate not on electrical signals, i.e., Boolean values, but real numbers.

Descriptive complexity is an area of computational complexity theory, that groups decision problems into classes not by bounds on the resources needed for their solution but by considering the syntactic complexity of a logical formalism able to specify the problems. Most well-known is probably Fagin’s characterization of the class NP as those problems that can be described by existential second-order formulas of predicate logic

[6]. Since then, many complexity classes have been characterized logically. Most important in our context is a characterization obtained by Neil Immerman, equating problems decidable by (families of) Boolean circuits of polynomial size and constant depth with those describable in first-order logic:

[[11]] .

An important issue in circuit complexity is uniformity, i.e., the question if a finite description of an infinite family of circuits exists, and if yes, how complicated it is to obtain it. Immerman’s Theorem holds both non-uniformly, i.e., under no requirements on the constructability of the circuit family, as well as for many reasonable uniformity conditions. In the non-uniform case, first-order logic is extended by allowing access to arbitrary numerical predicates, in symbols: non-uniform .

The rationale behind the descriptive approach to complexity is the hope to make tools from logic on expressive power of languages available to resource-based complexity and use non-expressibility results to obtain lower bounds on resources such as time, circuit size or depth, etc.

Descriptive complexity seems even more pertinent for real-valued computation than for computation on bit-strings, because formulas just like computation models over operate on structured inputs. Usually one considers metafinite structures, that is finite first-order structures enriched with a set of functions into another possibly infinite structure, in our case the real numbers . This study was initiated by Grädel and Meer [8], presenting logical characterizations of and . Continuing this line of research, Cucker and Meer showed a few logical characterizations for bounded fan-in real arithmetic circuit classes [4], which is what this paper builds on. Cucker and Meer first proved a characterization of using fixed-point logic, and building on this characterized the classes of the NC-hierarchy (bounded fan-in circuits of polynomial size and polylogarithmic depth) restricting the number of updating in the definition of fixed points to a polylogarithmic number. They leave out the case of the smallest interesting circuit class . We now expand on their research by making the framework of logics over metafinite structures amenable for the description of unbounded fan-in circuits; we are particularly concerned with a real analogue to the class and show that it corresponds to first-order logic over metafinite structures:

.

Cucker and Meer only note that ?the expressive power of first-order logic is not too big? since it can only describe properties in . In a sense we close the missing detail in their picture by considering the base-case.

The logical characterization of Theorem 1 holds for arbitrary uniformity conditions based on time-bounded construction of the circuit family, in particular P-uniformity and LT-uniformity. Extending the framework of Cucker and Meer, we also characterize non-uniform by first-order logic enhanced with arbitrary numerical predicates.

This paper is structured as follows: In the next section, we introduce the reader to machines and circuits over and the complexity classes they define. We also introduce logics over metafinite structures. Section 3 contains our results, first turning to non-uniform circuits and then generalizing our results to different uniformity conditions. We close by mentioning some questions for further work.

2 Preliminaries

In the upcoming section, we give an introduction to the machine models and logic over used in this paper – which are mostly taken from Cucker and Meer [4] – and some extensions thereof which we will make use of later on.

2.1 Machines over

Machines over , which were first introduced by Blum, Shub and Smale [1], operate on an unbounded tape of registers containing real numbers. We will define those machines as they have been defined by Cucker and Meer [4]. In order to describe these machines, we will use to denote arbitrarily long -vectors and for every such we use to denote its length. To talk about these machines’ tape contents we also consider the bi-infinite direct sum whose elementes have the form

where for all and for sufficiently large . For , we define the operations shift left and shift right which shift the indices of the elements of , e.g.

[Definition 1 [4]]

A machine over consists of an input space , an output space (for some ), a state space and a finite connected directed graph with nodes that are labelled for some and each of which has one of the following types:

There are only one input node and one output node. The input node is labelled with and is associated with its next node and the input map . The output node is labelled with . Once it is reached the computation halts and its result is placed in the output space by the output map . It therefore has no next nodes. A computation node is associated with a next node and a map where is a polynomial or rational map (i.e., a division of polynomials) on a finite number of coordinates and the identity on all others. A branch node is associated with two nodes: and . The next node of is if and otherwise. (Here denotes the zeroth coordinate of the vector representing the current state.) A shift node is associated with a next node and a map , where is either a left or a right shift.

The input map places an input in where the size of the input is stored in the zeroth coordinate. When the output space is , is the identity map on the first coordinates of , where is the number of consecutive ones stored in the negative coordinates of . If the output space is for some , we take as the identity restricted to the first coordinates of .

[Definition 1 [4]] For any given machine , we denote by the function which yields the output of when given an input and call that function the input-output function of . For any function , we say that is computable if there is a machine such that . Additionally, we say that a set is decidable if there is a machine

computing its characteristic function.

A -machine is said to work in time if for every input , reaches its output node after at most steps. Although generally, can be anything, in this paper we are mostly concerned with the case where is part of the input or its length.

We say that works in polynomial time if it works in time bounded by . Analogously, we say that works in logarithmic time if it works in time bounded by .

2.2 Arithmetic Circuits over

Arithmetic circuits over were first introduced by Cucker [3] and are our main model of computation. We will define them in analogy to how they were defined by Cucker and Meer [4], however in this paper we consider unbounded fan-in. Also, we disallow division and subtraction gates, since it can be shown that losing those gate types does not change computational power within polynomial size.

We define the function and one variation as follows:
  

Since and can be obtained from one another other, as and , we will use both freely whenever we have either one available.

An arithmetic circuit over is a directed acyclic graph. Its nodes (also called gates) can be of the following types:

Input nodes have indegree 0 and contain the respective input values of the circuit.
Constant nodes have indegree 0 and are labelled with real numbers.
Arithmetic nodes can have an arbitrary indegree only bounded by the number of nodes in the circuit. They can be labelled with either or .
Sign nodes have indegree 1.
Output nodes have indegree 1 and contain the output values of the circuit after the computation.

Nodes cannot be predecessors of the same node more than once, which leads to the outdegree of nodes in these arithmetic circuits being bounded by the number of gates in the circuit.

In order to later describe arithmetic circuits, we associate with each gate a number which represents its type. For a gate these associations are as follows:

input constant sign output
type 1 2 3 4 5 6

For convenience, we define auxiliary gates with types which do not grant us additional computational power as we show in Lemma 2.2. Those are arithmetic gates labelled with or the relation symbols , , , and . All of those nodes have indegree 2.

type 7 8 9 10 11 12

We will also refer to nodes of the types as relational nodes.

Arithmetic nodes compute the respective function they are labelled with, sign gates compute the sign function and relational nodes compute the characteristic function of their respective relation. On any input , a circuit computes a function by evaluating all gates according to their labels. The values of the output gates at the end of the computation are the result of ’s computation, i.e., .

In order to talk about complexity classes of arithmetic circuits, one considers the circuit’s depth and the size. The depth of a circuit is the longest path from an input gate to an output gate and the size of a circuit is the number of gates in a circuit.

We say that a directed acyclic graph is a subcircuit of a circuit , if and only if is weakly connected, all nodes and edges in are also contained in and it holds that if there is a path from an input gate to a gate in , then this path also exists in . For any node in , we denote by the subcircuit induced by that subcircuit of , of which is the top node. We then also say that is the root node of .

A single circuit can only compute a function with a fixed number of arguments, which is why we call arithmetic circuits a non-uniform model of computation. In order to talk about arbitrary functions, we need to consider circuit families, i.e., sequences of circuits which contain one circuit for every input length . The function computed by a circuit family is the function computed by the respective circuit, i.e.,

(1)

A circuit family is said to decide a set if and only if it computes the set’s characteristic function. For a function , we say that a circuit family is of size (depth ), if the size (depth) of is bounded by .

For any arithmetic circuit of polynomial size and constant depth which uses gates of the types , there exists an arithmetic circuit of polynomial size and constant depth computing the same function, which only uses gates of the types .

Proof.

Let be an arithmetic circuit with input gates which uses gates of the types , with and for . We will construct a circuit of polynomial size and constant depth which computes the same function as . We start out by and proceed as follows: First, since we can represent by

(2)

for all , we replace every gate in by a gate, followed by an addition gate, which in turn has a gate and a gate as its predecessors. Those two gates then each have the nodes and as their predecessors.The and addition gate at the top represent the in this construction. The overall increase in size is per gate, which leads to the overall increase in size being polynomial in the worst case. The increase in depth is at worst per gate on the longest path from an input gate to the output gate, which means that the overall increase in depth is constant. This means that still computes the same function as , its size is still polynomial and its depth is still constant in and now does not contain any gates. For gates, we proceed analogously. We continue similarly for the other cases: Since we can represent by

(3)

for all , we replace every gate in with predecessors and by a gate at the top, followed by an addition gate which in turn has a constant gate labeled and another gate as its predecessors. This construction represents . That second gate then has a subtraction gate as its predecessor, which has a constant node labeled and a gate as its predecessors. The gate has two gates as its predecessors, which in turn each have the same subtraction gate as their predecessor. That subtraction gate then has and as its predecessors. Note here that the gates here essentially work as identity gates, and we only need them, to have the value of be multiplied with itself in the node. The overall increase in size per gate in this construction is , which means that the total overhead in size is still polynomial in the worst case. In terms of depth, the increase is at worst per gate on the longest path from an input gate to the output gate, meaning that the total increase is still constant. After this step, computes the same function as , still has polynomial size and constant depth in and does not contain any gates. The construction for gates with predecessors and works similarly. We make use of being representable by

(4)

for all . We therefore replace every gate by a subtraction gate with and a construction for as above as its predecessors. The construction then has a subtraction gate as its predecessor, which in turn has the nodes and as its predecessors.The increase in size per gate is , leading to a polynomial increase at worst and the increase in depth is at worst per gate on the longest path from an input gate to the output, meaning that the overall overhead is constant. This means that still has polynomial size and constant depth in , still computes the same function as and now does not contain any gates. We proceed analogously for gates. For subtraction gates, we proceed similarly, since we can represent by

(5)

for all . We replace every subtraction gate with predecessors and by an addition gate with and a multiplication gate as its predecessors, where the multiplication gate has a constant node labeled and the node as its predecessors.For each gate, this introduces an increase in size of per subtraction gate, leading to the overall overhead still being polynomial in the worst case, and an increase in depth of for each gate on the longest path from an input gate to the output gate, which leads to the overall depth still being constant. Therefore, still computes the same function as , has polynomial size and constant depth in and does not contain any subtraction gates. In total, has polynomial size in , constant depth in , only contains gates of the types and computes the same function as . ∎

The class is the class of sets decidable by arithmetic circuit families over of polynomial size and constant depth. The circuit families we have just introduced do not have any restrictions on the difficulty of obtaining any individual circuit. This means that even with the strict constraints on the depth and size of the circuits for , it still contains some undecidable problems. For this reason, we also consider so-called uniform circuit families, i.e., families of circuits where the gates are numbered, the predecessors of each gate are ordered and where for any given triple of numbers , a corresponding triple can be computed by an -machine , where

  1. is the type of the th gate in ,

  2. is the number of the th predecessor of and

  3. is the value of if is a constant gate, the index if is the th input gate and otherwise.

If has less than predecessors, returns and if does not encode a gate in , returns .

If this computation only takes logarithmic time in , we call LT-uniform. If it takes polynomial time in , we call P-uniform.

For a circuit complexity class , we will by - denote the subclass of , which only contains sets definable by LT-uniform circuit families. We will use - to analogously denote those sets in definable by P-uniform families.

2.3 -structures and First-order Logic over

The logics we use to characterize real circuit complexity classes are based on first-order logic with arithmetics.

[Definition 7 [4]] Let , be finite vocabularies where can contain function and predicate symbols and only contains function symbols. An -structure of signature is a pair where is a finite structure of vocabulary which we call the skeleton of whose universe we will refer to as the universe of and whose cardinality we will refer to by . is a finite set which contains functions of the form for which interpret the function symbols in .

We will use Struct to refer to the set of all -structures of signature and we will assume that for any fixed signature , we can fix an ordering on the symbols in and .

In order to use -structures as inputs for machines, we encode them in as follows: We choose an arbitrary ranking on , i.e., a bijection . We then replace all predicates in by their respective characteristic functions and all functions by . Those functions are then considered to be elements of . We represent each of these functions by concatenating their function values in lexicographical ordering on the respective function arguments according to . To encode we only need to concatenate all representations of functions in in the order fixed on the signature. We denote this encoding by enc().

In order to be able to compute from enc(), we make an exception for functions and predicates of arity . We treat those as if they had arity , meaning that e.g. we encode a function as s.

Since

(6)

where is the arity of , we can reconstruct from the arities of the functions in and the length of . We can do so by using for example binary search, since we know that is between and . We can therefore compute when given and in time logarithmic in .

2.3.1 First-order Logic over

[First-order logic] The language of first-order logic contains for each signature a set of formulas and terms. The terms are divided into index terms which take values in the skeleton and number terms which take values in . These terms are inductively defined as follows:

  1. The set of index terms is defined as the closure of the set of variables under applications of the function symbols of .

  2. Any real number is a number term.

  3. For index terms and a -ary function symbol , is a number term.

  4. If , are number terms, then so are , and .

Atomic formulas are equalities of index terms and number terms , inequalities of number terms and expressions of the form , where is a k-ary predicate symbol and are index terms.

The set is the smallest set which contains the closure of atomic formulas under the Boolean connectives and quantification and where ranges over .

Equivalence of and sets defined by -formulas are done in the usual way, i.e., a formula defines a set if and only if the elements of are exactly the encodings of -structures under which holds and two such formulas are said to be equivalent if and only if they define the same set.

2.3.2 Extensions to

In the following, we would like to extend by additional functions and relations that are not given in the input structure. To that end, we make a small addition to Definition 2.3 where we defined -structures. Whenever we talk about -structures over a signature , we now also consider structures over . The additional vocabulary does not have any effect on the -structure, but it contains function and relation symbols, which can be used in a logical formula with this signature. This means that any -structure of signature is also a -structure of signature for any alphabet .

Let be a set of finite relations and functions. We will write FO to denote the class of sets for which there exists a -sentence over a signature such that for every input structure there is an interpretation such that with interpretation defines , where interprets each symbol in as a function or relation from .

With the goal in mind to create a logic which can define sets decided by circuits with unbounded fan-in, we introduce new rules for building number terms: the sum and the product rule. We will also give another rule, which we call the maximization rule, but will later show that we can define this rule in and thus do not gain expressive power by using it. We will use this rule to show that we can represent characteristic functions in .

[sum, product and maximization rule] Let be a number term in which the variable occurs freely with other variables and let denote the universe of the given input structure. Then

(7)

is also a number term which is interpreted as Moreover, and are defined analogously.

We will also write to denote for convenience and we will use analogously.

For a logic , we will by , and denote extended by the sum rule, the product rule or the maximization rule respectively.

We will now evaluate, which logics can already natively use some of the aforementioned rules. As it turns out, the maximization rule can be used in without any extensions and the sum and product rule extend neither nor a polynomial extension of which we will see later.

Proof.

Let be a formula which contains -constructions, i.e., number terms of the form for a number term . We will show that for every such formula, we can construct another -formula which is equivalent to but which does not contain the term . Since -constructions are number terms, whenever they occur, they are part of atomic (sub-)formulas. For this reason, we only need to show, how to turn atomic formulas with -constructions into semantically equivalent formulas (that are not necessarily atomic anymore). For a given atomic formula with -constructions , define as follows: Let and let be the -occurrences of , ordered by level of nesting, where has the lowest level of nesting, the nesting of is either the same as or greater by and so on. We assume without loss of generality that the variables and do not occur in . We also assume for now that there is only one occurrence of at the lowest level of nesting and that consists only of that outermost -construction, i.e., . To now construct , we go through the -occurrences in in reverse order of nesting, i.e., from the deepest level to the shallowest, and for each occurrence , we create a subformula , which ensures that is being maximized with respect to . We will use new variables in the subformulas, which will be quantified later, when we connect those subformulas to construct . will then have the form

(8)

where represents the structure of without any -constructions. In our case, would just be .

We start with the term , where is the number term in getting maximized by , are the variables used in from -constructions which occur at lower levels of nesting in and are all other variables used in .

We now create the subformula

(9)

which makes sure that is maximal with respect to .

Afterwards, we proceed in reverse order of nesting with the other -occurrences in (meaning that is next) and create the subformulas similarly. For , we proceed as follows:

Let be the occurrence of in with analogous as before. Now replace in all -constructions – where are all variables used in except for – by parentheses around , i.e., would just become . Denote the result by . We then define

(10)

Finally, we define

(11)

This construction now works for our strong assumption that . However, we only require the following modifications to make it generally applicable: If contains only one -construction at the lowest level, but then operates on that construction, we can just add the context of that -construction to the term in . For example if , then we could just add the ’’ to the in Formula 11. If contains several -constructions at the lowest level of nesting, then we can construct as we have previously and just add the subformulas to the conjunction in .

now does not contain any -constructions and is therefore a valid -formula. Since for every -occurrence in , there is a subformula in the conjunction of making sure that the term maximized by in is also maximal in , is also semantically equivalent to .

We can construct analogously, if both, and contain -constructions or if . We have therefore shown that for any -formula with -constructions, there exists a semantically equivalent formula which does not contain any such constructions. ∎

Let Arb denote the set of all finite relations and functions.

Proof.

In order to prove this, we will take an arbitrary -sentence in which number terms of the form occur and then create an -term which is semantically equivalent to but which does not contain any such constructions. Note that the set of the signature of will be different to that of . Let be a valid sentence of signature with the exception that it contains number terms of the form , where is a number term (which may again contain -constructions). Without loss of generality, we assume that for all number terms of the form , there is no symbol in the signature of . We now construct a number term which is equivalent to but which does not contain any constructions of the aforementioned form. We define step by step. We start out by . We now take any such instance of a number term in where itself does not contain any instances of -constructions. Let be the signature of . Without loss of generality, does not contain the symbol . Now add to the set of the function symbol of arity and replace in the term by . The interpretation of for any is the sum over all different values for if is in replaced by and is replaced by for all , i.e., for all :

(12)

The resulting formula is now semantically equivalent to , since we just moved the interpretation of the symbol to the function symbol , but it does not contain the instance of that we just removed. If we repeat this process for all remaining occurrences of in , we arrive at a sentence which is semantically equivalent to but which does not contain any instances of . can be shown analogously. ∎

For the sake of simplicity we will in the following only consider functional -structures, i.e., -structures whose signatures do not contain any predicate symbols. This does not restrict what we can express, since any relation can be replaced by its characteristic function .

As mentioned before, the reason why we need the maximization rule is that we would like to write characteristic functions as number terms. For a first-order formula we define its characteristic function on a structure by

(13)

The following result is a slight modification of a result presented by Cucker and Meer [4].

Let be a set of functions and predicates. For every FO-formula , there is a FO number term describing .

3 Characterizing

In this section, we give descriptive complexity results for the non-uniform set and some of its uniform subsets. In order to achieve this, we use the previously defined first-order logic over the real numbers and the extensions we defined.

3.1 A Characterization for non-uniform

First of all we show an equality which is close to a classical result shown by Immermann [10]. We show that extending our first-order logic over the reals with arbitrary functions lets us exactly describe the non-uniform set .

In the proof for the upcoming theorem, we make use of a convenient property of circuits deciding -sets, namely that for each of those circuits, there exist tree-like circuits deciding the same set. We call these circuits full trees.

For every -circuit family , there exists an -circuit family computing the same function, such that for all and every gate in , every path from an input gate to has the same length.

Proof.

In order to prove this, we construct, for any given -family , an -circuit family ’ which exhibits this property. Note that since we are talking about circuits deciding sets, we know that the given circuits will each only have one output gate. We give a generic construction for turning any circuit of into one of ’ which computes the same function. To achieve this, we proceed in two steps: for a given circuit we first create an equivalent circuit where only the input gates have an outdegree

, and which is thus very tree-like. Secondly, we will pad all paths from input gates to the output gate to have the same length. Due to the tree-likeness of our circuit, this property then translates to all nodes in the circuit.

Step one: Let be an -circuit which contains non-input gates with outdegree . We would like to get rid of those gates. To accomplish this, consider all subcircuits of induced by non-input gates which have outdegree in which every other non-input gate has outdegree . Since is acyclic, at least one such subcircuit must exist. These subcircuits are all distinct from each other, because only their respective root node has multiple successors (barring the input gates). For each of those subcircuits now proceed as follows: Let be the root node of . Replace each connection beyond the first from to a successor by a copy of , i.e., by a subcircuit which has the same input gates as and where all other gates and connections are copies of the gates and connections in . After this step, the longest distance between the output node and a non-input node with multiple successors whose induced subcircuit contains no non-input gates with successors is reduced by at least one. Repeat this process until there are no more non-input gates with multiple successors in and denote the circuit after the th step by . Let be such that . We show that the size of the circuit resulting in this process is still a polynomial in by induction. To be exact, we show that for all .

Base case : After the first step of this process we have increased the size of by less than for each of less than root nodes (because the output node cannot be such a root node) and each of less than successors thereof. This means that .

Induction step : In the th step, we replaced all subcircuits induced by non-input nodes with multiple successors in by copies. This means that all root nodes we consider in the th step have not been altered yet and that there are therefore less than of those. Additionally, since all those root nodes have multiple successors, no nodes reachable from these roots have been altered either. Therefore, each of those roots has less than successors. The subcircuits these nodes induce, however, have been altered and are therefore of size less than . After the th step, we have replaced less than subcircuits of size less than by less than copies each. Therefore it follows that .

Let denote the circuit after finishing the procedure above. Since we reduce the distance of the output node to the furthest such root node in each step, we only need to execute this process for a constant number of steps. Therefore for some , which means that the size of the is still a polynomial in . The depth of is still constant as the procedure we performed did not alter the circuits depth. Additionally, since we only added copies of subcircuits in place of subcircuits with several successors, we also did not change the computed function. This means that is still an -circuit which computes the same function as but does not contain any non-input gates with multiple successors.

Step two: We know that does not have any nodes with outdegree beyond the input gates. Consider now all paths from an input gate to the singular output gate. Let be the depth of , i.e., the length of the longest path from any input gate to the output gate. For every path now add successive addition gates in between the first node of – the respective input gate – and ’s second node. This ensures that all paths from input gates to the output gate have the same length. Denote the resulting circuit by . As we will see, this also results in the property that we wanted in the first place: for every node in , all paths from input gates to have the same length. We show this by contradiction: Assume that there is a gate in to which there are two paths from input gates with different lengths. We know that and all its successors have outdegree , therefore we know that there can be only one path from to the output node. That means that there would also be two paths of different length from input gates to the ouput node, which is a contradiction. As in step one, we have not added any depth to , but we increased its size. The increase in size, however, is less than , since there is at most one path from input to output for each outgoing edge of the input gates. There are input gates and there can be at most outgoing edges from those, so we have at most paths. Each path now gets padded by less than nodes. In the end, since is a polynomial in and is constant with respect to , the resulting circuit exhibits the properties that we desire and is still of constant depth and polynomial size. It also computes the same function as , since addition gates with only a singular predecessor are essentially just identity gates. ∎

.

Proof.

: To show that is included in , we will show that for any -sentence , we can create an circuit family which decides exactly the set defined by . Given a fixed size of input -structures ( = ), we can for any -formula reconstruct from as described on page 2.3. We will denote by . For any such formula with exactly free variables and for all we then create an arithmetic circuit with the following property: For any input structure it holds that if and only if is accepted by , where is substituted by for all . (That means that any such circuit for a formula outputs either or .) At the very top of the circuit is the output node. The rest of the circuit is defined by induction. A formula with free variables and natural numbers , with for all are given.

  1. Let . If does not occur free in , then . Otherwise, the free variables in are . now consists of a sign gate with an unbounded fan-in addition gate as its predecessor which in turn has the circuits as its predecessors for .

  2. If , then is defined as in the existential case, but with a multiplication gate below the sign gate.

  3. Let . Then consists of a subtraction gate, which subtracts the sign of from 1.

  4. Let . Then