A universal-algebraic proof of the complexity dichotomy for Monotone Monadic SNP

02/09/2018 ∙ by Manuel Bodirsky, et al. ∙ 0

The logic MMSNP is a restricted fragment of existential second-order logic which allows to express many interesting queries in graph theory and finite model theory. The logic was introduced by Feder and Vardi who showed that every MMSNP sentence is computationally equivalent to a finite-domain constraint satisfaction problem (CSP); the involved probabilistic reductions were derandomized by Kun using explicit constructions of expander structures. We present a new proof of the reduction to finite-domain CSPs which does not rely on the results of Kun. This new proof allows us to obtain a stronger statement and to verify the more general Bodirsky-Pinsker dichotomy conjecture for CSPs in MMSNP. Our approach uses the fact that every MMSNP sentence describes a finite union of CSPs for countably infinite ω-categorical structures; moreover, by a recent result of Hubička and Nešetřil, these structures can be expanded to homogeneous structures with finite relational signature and the Ramsey property. This allows us to use the universal-algebraic approach to study the computational complexity of MMSNP.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Monotone Monadic SNP (MMSNP) is a fragment of monadic existential second-order logic whose sentences describe problems of the form “given a structure , is there a colouring of the elements of that avoids some fixed family of forbidden patterns?” Examples of such problems are the classical -colourability problem for graphs (where the forbidden patterns are edges whose endpoints have the same colour), or the problem of colouring the vertices of a graph so as to avoid monochromatic triangles (Figure 1).

MMSNP has been introduced by Feder and Vardi [26], whose motivation was to find fragments of existential second-order logic that exhibit a complexity dichotomy between P and NP-complete. They proved that every problem described by an MMSNP sentence is equivalent under polynomial-time randomised reductions to a constraint satisfaction problem (CSP) over a finite domain, and conjectured that every finite-domain CSP is in P or NP-complete. Kun [34] later improved the result by derandomising the equivalence, thus showing that MMSNP exhibits a complexity dichotomy if and only if the Feder-Vardi dichotomy conjecture holds. Recently, Bulatov [20] and Zhuk [40] independently proved that the dichotomy conjecture indeed holds. Both authors establish a stronger form of the dichotomy, the so-called tractability conjecture, which gives a characterisation of the finite-domain CSPs that are solvable in polynomial time (assuming P is not NP). This characterisation is phrased in the language of universal algebra and is moreover decidable.

Figure 1: The No-monochromatic-triangle problem: the input is a finite graph , and the question is whether there exists a colouring of the vertices of with two colours that avoids monochromatic triangles.

The universal algebraic approach can also be used to study constraint satisfaction problems over infinite templates , at least if the structure is -categorical. If can even be expanded to a finitely bounded homogeneous structure, then there exists a generalisation of the tractability conjecture for finite-domain CSPs; see e.g. [4, 3, 1, 14]. Dalmau and Bodirsky [11] showed that every problem in MMSNP is a finite union of constraint satisfaction problems for -categorical structures. These structures can be expanded to finitely bounded homogeneous structures so that they fall into the scope of the mentioned infinite-domain tractability conjecture. It is easy to see that in order to prove the MMSNP dichotomy, it suffices to prove the dichotomy for those MMSNP problems that are CSPs (see Section 2.3). This poses the question whether the complexity of MMSNP can be studied directly using the universal-algebraic approach, rather than the reduction of Kun which involves a complicated construction of expander structures. In particular, even though we now have a complexity dichotomy for MMSNP, it was hitherto unknown whether the CSPs in MMSNP satisfy the infinite-domain tractability conjecture.

The main result of this paper is the confirmation of the infinite-domain tractability conjecture for CSPs in MMSNP. As a by-product, we obtain a new proof of the complexity dichotomy for MMSNP that does not rely on the results of Kun. To the best of our knowledge, this is the first-time that the universal-algebraic approach provides a classification for a class of computational problems that has been studied in the literature before111https://complexityzoo.uwaterloo.ca/Complexity_Zoo:M#mmsnp., and which has been introduced without having the universal-algebraic approach in mind. We also solve an open question by Lutz and Wolter [35]. Informally, we prove that the existential second-order predicates of an MMSNP sentence can be added to the original (first-order) signature of the sentence without increasing the complexity of the corresponding problem; we refer the reader to Section 5 for a formal statement.

Overview

Section 2 introduces MMSNP, CSPs, and how they relate. The choice of the template for a CSP in MMSNP is of course not unique, and the right choice of the infinite structures to work with in our analysis is one of the central topics in this article. In fact, there are differences between the infinite structures we work with on three levels:

  1. In certain proofs it is necessary to work with an expansion of the structure having a larger relational signature. We might expand the structure with unary relations that correspond to the monadic predicates of the MMSNP sentence. But we also need larger (first-order) expansions that make the structure homogeneous (see Section 2.2.4), or Ramsey (Section 4.1; this expansion is by a linear order which is not first-order definable). We finally also work with templates for MMSNP sentences where each monadic predicate extends a unary input predicate, called precoloured (Section 5), solving an open problem from [35].

  2. Even when we stick with the signature of our MMSNP sentence, the template is of course not unique. There always exists the up to isomorphism unique model-complete core template, but this is in many situations not the most appropriate template to work with; one of the reasons is, roughly speaking, that we sometimes need to work with injective polymorphisms with certain properties and that the model-complete core template might not have such polymorphisms.

  3. There is a third level of difficulties: not only do we care about the templates, but we also care about the description of the template. Different MMSNP -sentences might describe the same CSP. Which -categorical template we construct for an MMSNP sentence might not only depend on the CSP that is described by , but also on the sentence . Here we solve a problem that the last and first author have been discussing since 2005: we show that if is even in strong normal form (a concept from [37] that strengthens the MMSNP normal form introduced by Feder and Vardi [26, 38]), then the -categorical -structure that we obtain for is already the model-complete core template (Theorem 63).

One outcome of these investigations is the reduction of the classification to the precoloured situation, where the template also contains the monadic predicates of the MMSNP sentence in the input signature. The real classification work is then done in Section 6, and uses the following strategy:

  1. Using the infinite-to-finite reduction from [14], we show that a CSP in MMSNP is in P if the corresponding template has a canonical polymorphism that behaves on the orbits of the template as a Siggers operation.

  2. In order to prove that this is the only way to obtain polynomial-time tractability, we want to show that the absence of such a canonical polymorphism is equivalent to the existence of a uniformly continuous clonoid homomorphism to the clone of projections, which is known to entail NP-hardness [3]. We construct this map by first defining a clonoid homomorphism from the clone of canonical polymorphisms of the template to the clone of projections, followed by extending this map to the whole polymorphism clone (similarly as in [14]). For this, two ingredients are necessary.

  3. The first one is the fact that every polymorphism of the template locally interpolates a canonical operation. This requires proving that the template under consideration has an

    -categorical Ramsey expansion, which follows from recent results of Hubička and Nešetřil [31].

  4. The second ingredient is the fact that every polymorphism of our template canonises in essentially one way. We obtain this through an analysis of the binary symmetric relations that are preserved by the polymorphisms of the template.

This presentation of the strategy oversimplifies certain aspects, and we have to defer a more precise discussion to Section 6.

2 MMSNP and CSPs

We first formally introduce the logic MMSNP (Section 2.1). Our main result is not only the new proof of the dichotomy for MMSNP, but also the proof that the Bodirsky-Pinsker dichotomy conjecture holds for all CSPs in MMSNP; the dichotomy for all of MMSNP follows from this result. So we have to introduce CSPs, too, which will be done in Section 2.2. We then explain the connection between MMSNP and infinite-domain CSPs: first we syntactically characterise those MMSNP sentences that describe CSPs, by introducing the logic connected MMSNP, and then we show that the dichotomy for MMSNP and the dichotomy for connected MMSNP are equivalent (Section 2.3). In Section 2.4, we revisit the result of Dalmau and Bodirsky [11] that every connected MMSNP sentence is the CSP for an -categorical template.

2.1 Mmsnp

Let be a relational signature (we also refer to as the input signature). SNP is a syntactically restricted fragment of existential second order logic. A sentence in SNP is of the form where are predicates (i.e., relation symbols) and is a universal first-order-sentence over the signature . Monotone Monadic SNP without inequality, MMSNP, is the popular restriction thereof which consists of sentences of the form

where are monadic (i.e., unary) relation symbols not in , where is a tuple of first-order variables, and for every negated conjunct:

  • consists of a conjunction of atomic formulas involving relation symbols from and variables from ; and

  • consists of a conjunction of atomic formulas or negated atomic formulas involving relation symbols from and variables from .

Notice that the equality symbol is not allowed in MMSNP sentences.

Every MMSNP -sentence describes a computational problem: the input consists of a finite -structure , and the question is whether , i.e., whether the sentence is true in . We sometimes identify MMSNP with the class of all computational problems described by MMSNP sentences.

2.2 Constraint Satisfaction Problems

Let and be two structures with the same relational signature . A homomorphism from to is a map from (the domain of to (the domain of ) that preserves all relations. An embedding is a homomorphism which is additionally injective and also preserves the complements of all relations; in this case we write . For a relational -structure we write

  • for the class of all finite -structures that embed into ;

  • for the class of all finite -structures that homomorphically map into .

For example, is the 3-colouring problem: the signature is the signature of graphs, and denotes the clique with three vertices, i.e., .

Let be a class of finite relational -structures. We write

  • for the class of all finite -structures that do not embed a structure from ;

  • stands for the class of all finite -structures such that no structure in homomorphically maps to .

A relational structure is called finitely bounded if it has a finite signature and there exists a finite set of finite -structures (the bounds) such that .

2.2.1 Logic perspective

We present the classical terminology to pass from structures to formulas and vice versa. Let be a -structure. Then the canonical query of is the formula whose variables are the elements of , and which is a conjunction that contains for every a conjunct if and only if .

A primitive positive -formula (also known as conjunctive query in database theory) is a formula that can be constructed from atomic formulas using conjunction and existential quantification ; in other words, it is a first-order formula without using disjunction , negation , or universal quantification . By renaming the existentially quantified variables and pulling out the existential quantifiers, it is straightforward to rewrite primitive positive formulas into unnested formulas of the form

where are atomic -formulas, i.e., they are of the form or of the form where the variables might be from ; otherwise they are called free. We write if the free variables of are contained in . A formula without free variables is called a sentence.

Let be a primitive positive -formula without conjuncts of the form and written in the unnested form presented above. Then the canonical database of is the -structure whose elements are the variables of , and such that for every we have if and only if is a conjunct of . We will apply the notion of canonical database also to primitive positive formulas in general, by first rewriting them into unnested form and then applying the definition above. Since the rewriting might require that some of the existentially quantified variables are renamed, the resulting canonical database is not uniquely defined; but since we usually consider structures up to isomorphism, this should not cause confusions. Also note that the information which variable is existentially quantified and which variable is free is lost in the passage from a primitive positive formula to the canonical database. The following is straightforward and well-known.

Proposition 1 (See, e.g., [23]).

Let and be two structures. The following are equivalent.

  • has a homomorphism to .

  • where is the canonical query for and lists all the elements of .

2.2.2 PP-constructions

We say that two structures and with the same signature are homomorphically equivalent if there is a homomorphism from to and vice versa. A pp-power of is a structure with domain , for , whose -ary relations are primitive positive definable when viewed as -ary relations over . Let be a class of structures. We write

  • for the class of all structures that are homomorphically equivalent to structures in .

  • for the class of all structures obtained from structures in by taking pp-powers.

A structure is said to have a pp-construction over if it can be obtained from by repeated applications of and .

Lemma 2 ([3]).

Let be a relational structure with a finite relational signature. Then the structures with a pp-construction over are precisely the structures in . If then there is a polynomial-time reduction from to .

2.2.3 The finite-domain dichotomy theorem

We will use an important result from universal-algebra, Theorem 3 below; each of the equivalent items in this theorem will be used later in this article.

A polymorphism of a structure is a homomorphism from (a finite direct power of ) to . For every , , the projection given by is a polymorphism. The set of all polymorphisms of is denoted by ; this set forms a function clone, i.e., it is a set of operations on the set that is closed under composition and contains the projections. A map between two clones and that preserves the arities is called a clone homomorphism if for all -ary operations and all -ary operations . We write for the clone of projections on the set .

A set of functions is called a clonoid if for every of arity , every , and every , the function is in . Clearly, clones are clonoids. A map between two clonoids and that preserves the arities is called a clonoid homomorphism if for all -ary operations , , and .

Theorem 3 ([2, 22, 3]).

Let be a finite structure. Then the following are equivalent.

  1. contains .

  2. has no polymorphism of arity 6 which is Siggers, i.e., satisfies

  3. has no polymorphism of arity which is cyclic, i.e., satisfies

  4. There exists a clonoid homomorphism from to .

It is known that if a finite structure satisfies the equivalent items from Theorem 3, then is NP-hard [22]. Otherwise, we have the following recent result.

Theorem 4 (Finite-domain tractability theorem [20, 40]).

Let be a finite structure with finite relational signature which does not satisfy the equivalent conditions from Theorem 3. Then is in P.

2.2.4 Countable categoricity

Connected MMSNP sentences describe CSPs of countable structures that satisfy a strong property from model theory: -categoricity. A countably infinite structure is called -categorical if all countable models of the first-order theory of are isomorphic.

An endomorphism of is a homomorphism from to . The set of all endomorphisms of , denoted by , is a transformation monoid with respect to composition . An automorphism of is a bijective endomorphisms of such that is also an endomorphism of . The set of all automorphisms of , denoted by , forms a permutation group with respect to composition.

A structure is called homogeneous if every isomorphism between finite substructures of can be extended to an automorphism of . Homogeneous structures with finite relational signature are -categorical; this is a straightforward consequence of Theorem 5 below. A permutation group on a countably infinite set is called oligomorphic if for every there are finitely many orbits of -tuples on (with respect to the componentwise action of on ; this is often left implicit in the following).

Theorem 5.

A countable structure is -categorical if and only if is oligomorphic. In an -categorical structure, the orbits of the componentwise action of on are first-order definable in .

A finite or countably infinite -categorical structure is called a core if all endomorphisms of are embeddings, and it is called model-complete if all embeddings of into preserve all first-order formulas.

Theorem 6 ([7]).

Every -categorical structure is homomorphically equivalent to a model-complete core , which is up to isomorphism unique, -categorical, and embeds into .

The set of all maps from carries a natural topology, the topology of pointwise convergence, which is the product topology on where is taken to be discrete. We write for the closure of with respect to this topology. It is well-known (see e.g. Proposition 3.4.8 in [8]) that a subset of is closed if and only if for a structure on .

Proposition 7 ([7]).

For a countable -categorical structure , the following are equivalent.

  • is a model-complete core;

  • the orbits of tuples of the componentwise action of are primitive positive definable in ;

  • .

If is an -categorical model-complete core, then adding a unary singleton relation to does not change the computational complexity of . When is a class of relational structures, then is the class of all structures that can be obtained from a model-complete core in by adding a singleton unary relation. It is known (from [3]) that .

We also equip the set of all operations of finite arity on the set with a topology such that the polymorphism clones of relational structures with domain are precisely the closed subsets. The following result holds for all TODO

Theorem 8 ([4]).

Let be an -categorical model-complete core. Then either

  • has an expansion by finitely many unary singleton relations such that has a continuous clone homomorphism to , or

  • has no pseudo-Siggers polymorphism, i.e., a 6-ary polymorphism and unary polymorphisms and which satisfy

A map from a set of operations on a set to a set of operations on a set is uniformly continuous222There is indeed a natural uniformity on the set of all operations on a set that induces the topology that we have introduced earlier; but we do not need this further and refer to [3]. if and only if for all and all finite there exists a finite such that whenever two -ary functions agree on , then and agree on . In contrast to Theorem 8, the following theorem does not require that is a model-complete core (and this is one of the key points why this result becomes important later).

Theorem 9 ([3]).

Let be an -categorical structure. Then the following are equivalent.

  • contains .

  • has a uniformly continuous clonoid homomorphism to .

If these conditions apply, then is NP-hard.

For an -categorical model-complete core the conditions in Theorem 8 imply the conditions in Theorem 9, but the converse is false in general (see Theorem 1.6 in [3]). We will also need the following consequence of results from [3].

Proposition 10.

Let and at most countable -categorical structures with a homomorphism from to . Then there is a uniformly continuous clonoid homomorphism from to .

Proof.

In this proof, we use the terminology from [3], without repeating all the definitions here. Let and be the polymorphism clones of and , respectively. By Proposition 4.6 (iv) in [3], we have that . Let and be so that the reflection of by those functions is contained in . The map that sends to is a clonoid homomorphism (by Proposition 5.3 (iii); this is also straightforward to see) and clearly uniformly continuous: for any finite , if two -ary functions agree on , then and agree on . ∎

2.2.5 The infinite-domain dichotomy conjecture

There are -categorical model-complete cores (even homogeneous digraphs) that do not satisfy the conditions from Theorem 9 but is even undecidable [16]. So to generalise the finite-domain tractability theorem we consider a subclass of the class of all -categorical structures, namely structures that are homogeneous and finitely bounded. More generally, we also consider first-order reducts of such structures, i.e., structures with the same domain as a homogeneous finitely bounded structure such that all relations of are first-order definable over . For such structures, Bodirsky and Pinsker conjectured the following pendant to the finite-domain tractability conjecture.

Conjecture 11 (Infinite-domain tractability conjecture; see e.g. [18]).

Let be a first-order reduct of a finitely bounded homogeneous structure with finite relational signature. If the conditions in Theorem 8 apply then is in P.

For first-order reducts of homogeneous structures with finite signature it has been shown in [1] that the items in Theorem 8 are equivalent to the items in Theorem 9 (as in the finite).

Theorem 12 (Corollary 1.8 in [1]).

Let be a first-order reduct of a homogeneous structure with finite relational signature. Then the following are equivalent.

  • There is an expansion of the model-complete core of by finitely many unary singleton relations such that has a continuous clone homomorphism to .

  • has a uniformly continuous clonoid homomorphism to .

It is an open problem whether the uniform continuity requirement can be dropped in this theorem.

2.3 Connected MMSNP

A primitive positive formula with at least one variable is called connected if the conjuncts of cannot be partitioned into two non-empty sets of conjuncts with disjoint sets of variables, and disconnected otherwise. Note that a primitive positive formula without equality conjuncts is connected if and only if the Gaifman graph333The Gaifman graph of a relational structure is the undirected graph with vertex set which contains an edge between if and only if and both appear in a tuple contained in a relation of . of the canonical database of is connected in the graph theoretic sense. A connected primitive positive formula is called biconnected if the conjuncts of cannot be partitioned into two non-empty sets of conjuncts that only share one common variable. Note that formulas with only one variable might not be biconnected, e.g., the formula is not biconnected. An MMSNP -sentence is called connected (or biconnected) if for each conjunct of where is a conjunction of -formulas and is a conjunction of unary formulas, the formula is connected (or biconnected, respectively).

Proposition 13 (implicit in [26]; see also Section 6 of [38]).

Let be an MMSNP sentence. Then is logically equivalent to a finite disjunction of connected MMSNP sentences; these connected MMSNP sentences can be effectively computed from .

Proof.

Let be the existential monadic predicates in , and let be the input signature of . Suppose that has a conjunct where is a disconnected conjunction of atomic -formulas and contains unary predicates only. Suppose that is equivalent to for non-empty formulas and . Let be the MMSNP sentence obtained from by replacing by , and let be the MMSNP sentence obtained from by replacing by . It is then straightforward to check that every finite -structure we have that satisfies the first-order part of if and only if satisfies the first-order part of or the first-order part of . Iterating this process for each disconnected clause of , we eventually arrive at a finite disjunction of connected MMSNP sentences. ∎

It is well-known that complexity classification for MMSNP can be reduced to complexity classification for connected MMSNP; we add the simple proof for the convenience of the reader.

Proposition 14.

Let be an MMSNP -sentence which is logically equivalent to for connected MMSNP -sentences where is smallest possible. Then is in P if each of is in P. If one of the is NP-hard, then so is .

Proof.

If each can be decided in polynomial time by an algorithm , then it is clear that can be solved in polynomial time by running all of the algorithms on the input, and accepting if one of the algorithms accepts.

Otherwise, if one of the describes an NP-complete problem, then can be reduced to as follows. Since was chosen to be minimal, there exists a -structure such that satisfies , but does not satisfy for all that are distinct from , since otherwise we could have removed from the disjunction without affecting the equivalence of the disjunction to . We claim that satisfies if and only if satisfies . First suppose that satisfies . Since also satisfies by choice of , and since is closed under disjoint unions, we have that satisfies as well. The statement follows since is a disjunct of .

For the opposite direction, suppose that satisfies . Since does not satisfy for all distinct from , does not satisfy as well, by monotonicity of . Hence, must satisfy . By monotonicity of , it follows that satisfies . Since is for fixed clearly computable from in linear time this concludes our reduction from to . ∎

Proposition 15 (Corollary 1.4.15 in [9]).

An MMSNP sentence describes a CSP if and only if is logically equivalent to a connected MMSNP sentence.

2.4 Templates for connected MMSNP sentences

In this section we first revisit the fact that every connected MMSNP sentence describes a CSP of an -categorical structure [11]. The proof uses a theorem due to Cherlin, Shelah, and Shi, stated for graphs in [24]; Theorem 16 below is formulated for general relational structures. Another proof of the theorem of Cherlin, Shelah, and Shi has been given by Hubička and Nešetřil [29].

A structure does not have algebraicity if for all first-order formulas with free variables , and all elements of the set

is either infinite or contained in ; otherwise, we say that the structure has algebraicity. It is well-known that a homogeneous structure has no algebraicity if and only if its age has strong amalgamation, i.e., if for any two finite substructures and of there exists a substructure of and embeddings and such that .

Theorem 16 (Theorem 4 in [24]).

Let be a finite set of finite connected -structures. Then there exists a countable model-complete -structure such that . The structure is up to isomorphism unique, -categorical, and without algebraicity.

Let be a connected MMSNP -sentence. Let be the existentially quantified unary relation symbols in , and let be the signature that contains a relation symbol for every relation symbol . We write for the maximal number of variables in the clauses of . For every , add the clause to . Let be the formula obtained from by replacing each occurrence of in by . Then the obstruction set for is the set of all finite connected -structures such that

  • for ;

  • for every either or holds;

  • falsifies a clause of .

Note that satisfies the conditions from Theorem 16.

Definition 17.

Let be an MMSNP sentence, and the obstruction set for . Then denotes the substructure induced in by all the elements such that for all .

Let be a subset of the signature of ; then the -reduct of is the -structure obtained from by dropping all relations that are not in , and denoted by . Note that reducts of -categorical structures are -categorical, and hence the structure is -categorical for all .

Theorem 18 ([11]).

Let be an MMSNP -sentence. Then a finite -structure satisfies if and only if homomorphically maps to .

2.5 Statement of the main result

The main result of this article is the proof of the infinite-domain tractability conjecture (Conjecture 11) for CSPs in MMSNP. We actually show a stronger formulation than the conjecture since we also provide a characterisation of the polynomial-time tractable cases using pseudo-Siggers polymorphisms (which does not follow from Theorem 8 since the structures under consideration need not be model-complete cores).

Combined with Proposition 13 we obtain the following theorem for MMSNP in general.

Theorem 19.

Let be an MMSNP -sentence. Then is logically equivalent to a finite disjunction of connected MMSNP sentences; for each there exists an -categorical structure such that describes , and either

  • has a uniformly continuous clonoid homomorphism to , for some , and is NP-complete.

  • contains a pseudo-Siggers polymorphism, for each , and is in P.

In particular, every problem in MMSNP is in P or NP-complete.

3 Normal Forms

We recall and adapt a normal form for MMSNP sentences that was initially proposed by Feder and Vardi in [25, 26] and later extended in [38]. The normal form has been invented by Feder and Vardi to show that for every connected MMSNP sentence there is a polynomial-time equivalent finite-domain CSP. In their proof, the reduction from an MMSNP sentence to the corresponding finite-domain CSP is straightforward, but the reduction from the finite-domain CSP to is tricky: it uses the fact that hard finite-domain CSPs are already hard when restricted to high-girth instances. The fact that MMSNP sentences in normal form are biconnected is then the key to reduce high-girth instances to the problem described by .

In our work, the purpose of the normal form is the reduction of the classification problem to MMSNP sentences that are precoloured in a sense that will be made precise in Section 5, which is later important to apply the universal-algebraic approach. Moreover, we describe a new strong normal form that is based on recolourings introduced by Madelaine [36]. Recolourings have been applied by Madelaine to study the computational problem whether one MMSNP sentence implies another. In our context, the importance of strong normal forms is that the templates that we construct for MMSNP sentences in strong normal form, expanded with the inequality relation , are model-complete cores (Theorem 63). Let us mention that in order to get this result, the biconnectivity of the MMSNP sentences in normal form is essential (e.g, the proof of Theorem 63 uses Corollary 36, which uses Lemma 34, which uses Lemma 25, which crucially uses biconnectivity of ).

3.1 The normal form for MMSNP

Every connected MMSNP sentence can be rewritten to a connected MMSNP sentence of a very particular shape, and this shape will be crucial for the results that we prove in the following sections.

Definition 20 (originates from [26]; also see [38]).

Let be an MMSNP sentence where , for , are the existentially quantified predicates (also called the colours in the following). Then is said to be in normal form if it is connected and

  1. (Every vertex has a colour) the first conjunct of is

  2. (Every vertex has at most one colour) contains the conjunct

    for all distinct ;

  3. (Clauses are fully coloured) for each conjunct of except the first, and for each variable that appears in , there is an such that has a literal of the form ;

  4. (Clauses are biconnected) if a conjunct of is not of the form as described in item 1 and 2, the formula is biconnected;

  5. (Small clauses are explicit) any -structure with at most elements satisfies the first-order part of if satisfies all conjuncts of with at most variables.

Note that when is in normal form then in all conjuncts of except for the first we can drop conjuncts where predicates appear negatively in ; hence, we assume henceforth that is a conjunction of atomic formulas. We illustrate item 4 and item 5 in this definition with the following examples.

Example 21.

Let be the connected MMSNP sentence

which is in fact a first-order formula. The canonical database of

has only four elements, does not satisfy , but the only conjunct of has five elements. So this is an example that satisfies all items except item 5 in the definition of normal forms.

However, is logically equivalent to the following MMSNP formula, and it can be checked that this formula is in normal form.

Adding clauses to an MMSNP sentence to obtain an equivalent sentence that satisfies item 5 can make a biconnected sentence not biconnected, as we see in the following example.

Example 22.

Let be the following biconnected MMSNP sentence.

Note that does not satisfy item 5 (it has implicit small clauses) and in fact is equivalent to

which is not biconnected.

Lemma 23.

Every connected MMSNP sentence is equivalent to an MMSNP sentence in normal form, and can be computed from .

Proof.

We transform in several steps (their order is important).

1: Biconnected clauses.

Suppose that contains a conjunct such that is not biconnected, i.e., can be written as for tuples of variables and with disjoint sets of variables, and where and are conjunctions of atomic formulas. Then we introduce a new existentially quantified predicate , and replace by . Repeating this step, we can establish item 4 in the definition of normal forms.

2: Making implicit small clauses explicit.

Let be a conjunct of that is not the first conjunct. Let be a variable that does not appear among , and consider the formula where is either or , and suppose that for at least two different . If is biconnected, then add to . Otherwise, can be written as . We then apply the procedure from step 1 with the formula . In this way we can produce an equivalent MMSNP sentence that still satisfies item 4 (biconnected clauses). When we repeat this in all possible ways the procedure eventually terminates, and we claim that the resulting sentence satisfies additionally item 5. To see this, let be a -structure with at most elements which does not satisfy some conjunct of . Pick the conjunct from with the least number of variables and this property. Then there are such that satisfies . If , we are done. Otherwise, there must be such that . If the conjunct is biconnected, it has been added to , and it has less variables than , a contradiction. Otherwise, our procedure did split the conjunct, and inductively we see that a clause that it not satisfied by and has less variables than has been added to .

3: Predicates as colours.

Next, we want to ensure the property that contains for each pair of distinct existentially quantified monadic predicates the negated conjunct

and when are all the existentially quantified predicates, then contains the negated conjunct

We may transform every MMNSP sentence into an equivalent MMSNP sentence of this form, via the addition of further monadic predicates ( predicates starting from monadic predicates). If then was a first-order formula; in this case, to have a unified treatment of all cases, we introduce a single existentially quantified predicate , too.

4: Fully coloured clauses.

Finally, if is a conjunct of and a variable from such that does not appear in any literal of the form in , then we replace by the conjuncts

We do this for all conjuncts of and all such variables, and obtain an MMSNP sentence that finally satisfies all the items from the definition of normal forms. ∎

Example 24.

We revisit an MMSNP sentence from Example 22,

An equivalent MMSNP sentence in normal form is

The following lemma states a key property that we have achieved with our normal form (in particular, we use the biconnectivity assumption).

Lemma 25.

Let be the first-order part of an MMSNP -sentence in normal form with color set and let and be two conjunctions of atomic -formulas such that

  • and

    are vectors of disjoint sets of variables;

  • the canonical databases of and of satisfy ;

  • the canonical database of does not satisfy .

Then must contain a literal and must contain a literal for distinct colours and of .

Proof.

First observe that all vertices of must be coloured since all vertices of the canonical databases of and of are coloured (because they satisfy ). Therefore, since does not satisfy , there is a conjunct of and such that . Pick the conjunct such that is minimal. Since both the canonical database of and of satisfy , not all of can lie in the canonical database of , or in the canonical database of . If is of the form for then we are done. Otherwise, since is biconnected, there are such that . In this case, the structure induced by in has strictly less then elements. Since is in normal form, and since does not satisfy , by item 5 in the definition of normal forms there must be a conjunct of with at most variables such that holds in . This contradicts the choice of . ∎

3.2 Templates for sentences in normal form

Let be an MMSNP -sentence in normal form. Let be the set of colours of . We will now construct an -categorical -structure for an MMSNP sentence in normal form; this structure will have several important properties:

  1. a structure satisfies if and only if homomorphically maps to ;

  2. has no algebraicity;

  3. the colours of are in bijective correspondence to the orbits of ;

  4. is a model-complete core;

  5. if is furthermore in strong normal form (to be introduced in Section 3.4) then even is a model-complete core.

If is an MMSNP sentence in normal form, it is more natural to consider a variant of the notion of an obstruction set introduced in Section 2, which we call coloured obstruction set, because when is in normal form we do not have to introduce a new symbol for the negation of each existentially quantified predicate to construct a template.

Definition 26.

Let be an MMSNP -sentence in normal form. The coloured obstruction set for is the set of all canonical databases for formulas such that is a conjunct of , except for the first conjunct.

Theorem 16 has the following variant in the category of injective homomorphisms.

Theorem 27.

Let be a finite set of finite connected -structures. Then there exists a -structure such that

  • a finite -structure homomorphically and injectively maps to if and only if ;

  • is a model-complete core.

The structure is unique up to isomorphism, has no algebraicity, and is -categorical.

Proof.

Let be the model-complete core of ; by Theorem 6 the structure is unique up to isomorphism, and -categorical. Let be a finite -structure. If , then embeds into by Theorem 16, and since is homomorphically equivalent to , there is an injective homomorphism from to . These reverse implication can be shown similarly, and this shows the first item.

For proving that has no algebraicity, let be a first-order -formula and be elements of . By Theorem 6 we can assume that is a substructure of . Since is a model-complete core, the formula is equivalent to an existential positive -formula over . Suppose that the set contains an element . Then

and since does not have algebraicity, the set must be infinite. Let be a homomorphism from to . Since preserves we have that is infinite, and since preserves the existential positive formula we have , which proves that is infinite. ∎

The structure from Theorem 16 and the structure from Theorem 27 might or might not be isomorphic, as we see in the following example.

Example 28.

The structure might be isomorphic to the structure : it is for example easy to verify that for the structure is a model-complete core, and therefore isomorphic to .

In general, however, the two structures are not isomorphic. Consider for example the signature for binary and where , i.e., is the canonical database of . Then all finite -structures embed into , but satisfies , i.e., is the countably infinite clique.

Definition 29.

Let be an MMSNP -sentence in normal form and let be the coloured obstruction set of . Then denotes the substructure of induced by the coloured elements of .

The -reduct of the structure that we constructed for an MMSNP sentence in normal form is indeed a template for the CSP described by .

Lemma 30.

Let be an MMSNP -sentence in normal form and let be a -structure. Then the following are equivalent.

  1. ;

  2. homomorphically and injectively maps to ;

  3. homomorphically maps to .

Proof.

Let be the colour set and let be the coloured obstruction set of . . If satisfies it has a -expansion such that no structure in homomorphically maps to . So homomorphically and injectively maps to by Theorem 27. Moreover, every element of is contained in one predicate from (because of the first conjunct of ) and hence the image of the embedding must lie in .

is trivial. For , let be the homomorphism from to . Expand to a -structure by colouring each element by the colour of in ; then there is no homomorphism from a structure to , since the composition of such a homomorphism with would give a homomorphism from to , a contradiction. The expansion also satisfies the first conjunct of , and hence . ∎

In the following we prove that indeed has the properties that we announced at the beginning of this section. We start with some remarkable properties of the structure (Section 3.2.1) and continue with properties of (Section 3.2.2).

3.2.1 Properties of Cherlin-Shelah-Shi structures

An existential formula is called primitive if it does not contain disjunctions.

Lemma 31.

For every , the orbits of -tuples in can be defined by where is a primitive positive formula and is a conjunction of negated atomic formulas.

Proof.

It suffices to prove the statement for -tuples with pairwise distinct entries. Since is -categorical and model-complete, there is an existential definition of the orbit of in . Since defines an orbit of -tuples it can be chosen to be primitive. Moreover, since is a tuple with pairwise distinct entries, can be chosen to be without conjuncts of the form (it is impossible that both and are among the free variables ; if one of the variables is existentially quantified, we can replace all occurrences of it by the other variable and obtain an equivalent formula). Let be the primitive positive formula obtained from by deleting all the negated conjuncts. Let be conjunction of all negated atomic formulas that hold on . Clearly, implies .

Let be a tuple that satisfies ; we have to show that satisfies . Let be the existential definition of the orbit of . Again, we may assume that is disjunction-free and free of literals of the form . Let be the formula obtained from by dropping negated conjuncts. Let be the canonical database of (which is well-defined since both and are primitive positive and do not involve literals of the form ). We have