Among the variants of graph coloring, there is a prominent family of problems, where the coloring of vertices of is not given explicitly, but derived from some other function. Usually, this function is some labeling of edges or vertices of , and the color of the vertices is based on the labels assigned to incident edges or adjacent vertices (or both in case of total labeling).
Perhaps the most famous problem of this kind was proposed by Karoński, Łuczak, and Thomason . They were labeling the edges of a graph with integers , so that for every two adjacent vertices , the sums of labels assigned to edges incident to each of and are different. We call such a labeling neighbor sum distinguishing. Observe a neighbor sum distinguishing labeling exists if and only if has no isolated edge, such graphs will be called nice. Karoński, Łuczak, and Thomason showed that each nice graph has a neighbor sum distinguishing labeling with 183 labels, if we allow real numbers as labels. They also showed that if the minimum degree of is large, 30 (real) labels suffice, and conjectured that every nice graph has a neighbor sum distinguishing labeling with labels . This problem, known as the 1-2-3 conjecture, raised significant interest in the graph theory community. Addario-Berry, Dalal, McDiarmid, Reed, and Thomason  showed that integer labels are sufficient to find a neighbor sum distinguishing labeling of any nice graph. The upper bound on the largest label was subsequently improved: to 16 by Addario-Berry, Dalal, and Reed , then to 13 by Wang and Yu , and then to 6 by Kalkowski, Karoński, and Pfender . Currently best bound for general graphs is 5 and was shown by Kalkowski, Karoński, and Pfender . Bartnicki, Grytczuk, and Niwczyk  proposed a stronger conjecture, that for any assignment of 3-element lists to edges of , one can find a neighbor sum distinguishing labeling, such that every edge gets a label from its list. A constant bound on the size of lists that guarantee the existence of such a labeling is known for some special classes of graphs, e.g. complete graphs, complete bipartite graphs, or nice trees .
Dudek and Wajc  considered the computational problem of deciding whether a given nice graph has a neighbor sum distinguishing edge labeling with labels . They proved that the problem is NP-complete for and for . This was later extended by Dehghan, Sadeghi, and Ahadi  to all pairs , even if the input graph is cubic.
Let us have a closer look at the already mentioned result by Karoński, Łuczak, and Thomason , that a constant number of real labels is sufficient to distinguish adjacent vertices by sums of labels on incident edges. We will say that an edge labeling distinguishes neighbors by multisets, if for every pair of adjacent vertices, the multisets of labels appearing on the edges incident to and are different, see Figure 1 a). Notice that different sums always imply different multisets, but it is possible to have different multisets that give the same sum. The authors of  proved that every graph has an edge labeling, which distinguishes neighbors by multisets, and uses a constant number of labels and then, by choosing labels that satisfy certain independence properties, one can obtain a labeling that distinguishes sums. Edge labelings distinguishing neighbors by multisets were further studied by Addario-Berry, Aldred, Dalal, and Reed , who showed every nice graph has such a labeling using four labels and, if the minimum degree of is large enough, three labels suffice. Observe that if is regular, then an edge labeling using two labels is neighbor sum distinguishing if and only if it distinguishes neighbors by multisets. Thus, by the already mentioned result by Dehghan, Sadeghi, and Ahadi , it is NP-complete to decide whether a given graph has an edge labeling distinguishing neighbors by multisets.
Instead of considering sums or multisets of labels appearing on edges incident to adjacent vertices, one can also consider sets. We say that an edge labeling distinguishes neighbors by sets if for any two adjacent vertices and , the sets of labels on edges incident to and are different, see Figure 1 b). Observe that different sets of colors always imply different multisets, but not the other way around. On the other hand, sets are sums are incomparable: one may have different multisets that give the same sum and different sets, or the same set and different sums. By the generalized neighbor-distinguishing index of a graph , denoted by , we mean the minimum number of labels in an edge labeling distinguishing neighbors by sets. This parameter was introduced by Győri, Horňák, Palmer, and Woźniak , who proved that for every nice graph we have . This bound was later refined by Horňak and Sotak  and by Győri and Palmer , who proved that if , then . Horňák and Woźniak considered a list variant of the problem, where each edge is equipped with a list of possible labels and we ask for the existence of a labeling distinguishing neighbors by multisets and respecting these lists. They proved tight bounds on the size of lists, that guarantee the existence of such a labeling in paths and cycles, and showed that for trees lists of size three are sufficient. They also showed that lists of size three may be necessary, even for trees with .
Inspired by the 1-2-3 conjecture, Woźniak and Przybyło  suggested a closely related problem considering total labelings of , i.e., labelings of edges and vertices. They considered the problem of finding a total labeling using minimum number of labels, which distinguishes adjacent vertices by sums of labels appearing on incident edges and the vertex itself. Such a labeling is called neighbor sum distinguishing total labeling. Woźniak and Przybyło conjectured that every graph has a neighbor sum distinguishing total labeling, using labels only, this problem is known as the 1-2 conjecture. Kalkowski  showed that each graph has a neighbor sum distinguishing total labeling with labels , in which the label 3 does not appear on any vertex. Wong and Zhu , and Przybyło and Woźniak  conjectured that the 1-2 conjecture holds even in the list variant. Recently, Wong and Zhu  showed a list version of the theorem by Kalkowski: a list neighbor sum distinguishing total labeling exists if each vertex has a list of size 2 and every edge has a list of size 3. Observe that in an analogous way one may define total labelings distinguishing neighbors by multisets and distinguishing neighbors by sets, see Figure 2.
Finally, let us mention a similar problem, considered by Seamone and Stevens . Let be a graph and suppose that its edges are linearly ordered. A labeling of edges distinguishes neighbors by sequences, if the sequences of colors (implied by the global ordering of edges), appearing on edges incident to adjacent vertices, are different.
Seamone and Stevens showed that if the ordering of edges can be chosen, then for any nice graph lists of size 2 are sufficient to find a list edge labeling, distinguishing neighbors by sequences. If the ordering of edges is fixed, the lists of size 3 suffice, provided that the minimum degree is large enough, compared to the maximum degree. In particular, lists of size 3 are sufficient for a -regular graph with .
Another variants of the mentioned problems have also been studied. For example, one can ask for an edge-labeling, which distinguishes neighbors by sums/multisetes/sums, and is also required to be proper: for distinguishing neighbors by sums of labels, see e.g. Przybyło [47, 48], Bonamy and Przybyło , Hocquard and Przybyło ; for distinguishing neighbors by sets of labels, see Zhang, Liu, and Wang , Balister, Győri, Lehel, and Schelp , Edwards, Horňák, and Woźniak , Bonamy, Bousquet, and Hocquard  or Hatami ; for list edge labelings distinguishing neighbors by multisets see a recent exciting result by Kwaśny and Przybyło .
There is also some work on edge labelings, that distinguish neighbors by products of labels (see Skowronek-Kaziów [55, 56, 57]). For a more detailed overview on the related problems, we refer the reader to the recent book by Zhang , and a survey by Seamone .
1.1 Our contribution
Let us introduce the main character of this paper. For a hypergraph , we say that a vertex coloring is intersecting edge distinguishing by multisets (or, in short, is an iedm-coloring), if the multisets of colors appearing in intersecting edges are different. Similarly, a vertex coloring is intersecting edge distinguishing by sets (or, in short, is an ieds-coloring), if the sets of colors appearing in intersecting edges are different.
It is perhaps interesting to mention the special case of graphs, i.e., 2-uniform hypergraphs. It is straightforward to verify that in this case iedm- and ieds-colorings are equivalent. Moreover, a vertex coloring distinguishes intersecting edges by sets (or, equivalently, multisets) if and only if no two vertices with a common neighbor receive the same color. Such a concept is already known in graph theory and usually referred to as an -labeling. The motivation to study this kind of a coloring came from the hidden terminal problem in telecommunication [10, 42]. Optimal (i.e., using the minimum number of colors) -labelings are known for simple classes of graphs, like paths, cycles, grids (see Makansi  and Jin, Yeh ), hypercubes (see Wan ), and complete binary trees (see Bertossi and Bonuccelli ). Bodlaender, Kloks, Tan, van Leeuwen  showed some bounds on the number of colors required to find an -labeling of for some special classes of graphs: bounded-treewidth graphs, permutation graphs, outerplanar graphs, split graphs, and bipartite graphs. On the complexity side, it is known that the decision problem whether an input graph has an -labeling with 3 colors is NP-complete for planar graphs  and split graphs . The parameterized complexity of this problem was considered by Fiala, Golovach, and Kratochvíl , who showed that the problem is -hard, when parameterized by treewidth, but FPT, when parameterized by the vertex cover number (we refer the reader to the book by Cygan et al.  for more information about parameterized complexity).
In Section 2.2 we argue that colorings of hypergraphs, that distinguishing intersecting edges by sets or multisets, are natural common generalization of edge and total labelings distinguishing neighbors by sets and multisets. We are interested in the list variants of both problems. As the main result, in Theorem 1 we show upper bounds on the size of lists that guarantee the existence of a list ieds-coloring or a list iedm-coloring of a given hypergraph.
The main part of the paper, i.e., Section 3, is devoted to the proof of Theorem 1. The proof uses the so-called entropy compression method, which is a variant of the Lovász Local Lemma . The Local Lemma is essentially non-constructive, but several algorithmic versions have also been developed (see Alon , Molloy, Reed , and Moser, Tardos ). Entropy compression originates in the algorithmic version of the Local Lemma by Moser and Tardos, and was first used by Grytczuk, Kozik, and Micek  to study the list version of the problem of Thue. Then the method was successfully applied in many other contexts (see e.g. Esperet, Parreau  or Dujmović, Joret, Kozik, and Wood ). The main idea of our proof is similar to the one of Bosek, Czerwiński, Grytczuk, and Rzążewski , but there are two significant differences. First, the authors of  considered the so-called harmonious colorings, where all pairs of edges need to be distinguished. Second, they were considering colorings in which no edge contained two vertices in the same color. If we drop this restriction, distinguishing edges by sets is significantly more difficult and makes the argument more complicated.
The proof of Theorem 1 is essentially a randomized algorithm which finds a list ieds- or iedm-coloring of a given hypergraph. In Section 3.4 we consider its computational complexity and show that the expected number of steps in the execution of the algorithm is polynomial.
In Section 4 we discuss the applicability of Theorem 1 and the method used in the proof. As corollaries from Theorem 1, in Section 4.1 we obtain several bounds on the size of lists, which guarantee the existence of a list edge/total labeling, distinguishing neighbors of a regular graph by sets or multisets. In particular, in Corollaries 8 and 9 we show that if is sufficiently large, then lists of size are sufficient to find an edge or total labeling distinguishing neighbors by sets. In case of distinguishing by multisets, in Corollaries 10 and 11 we prove that lists of size suffice, if is large enough. We also consider the case of the so-called configurations. In Section 4.2, we show that our method can be easily extended and adapted to other problem of similar kind. In particular, we show how to construct an algorithm, which finds a list coloring of vertices of a hypergraph, distinguishing intersecting edges by sequences. We omit most of the details of the proof, as it is essentially the same as the proof of Theorem 1. We focus on highlighting the crucial issues that have to be considered, when adapting our approach to a new problem. As a side result, we improve the result of Seamone and Stevens  for regular graphs. In particular, we show that given a -regular graph with , with a fixed ordering of edges, then lists of size 2 are sufficient to choose a list edge labeling, in which every two adjacent vertices have distinct sequences of colors appearing on incident edges.
In Section 5, we investigate an interesting relation between the generalized neighbor-distinguishing index of bipartite graphs and two well-known problems, i.e., a variant of the satisfiability problem called Not-All-Equal Sat, and the so-called property B of hypergraphs. The paper is concluded in Section 6 with several open problems and suggestions for future work.
For an integer , by we denote the set . By we denote the number of total preorders of . In other words, is the number of orderings of with possible ties. Observe that
where denotes the Stirling number of the second kind, i.e., the number of partitions of into non-empty subsets. The value is sometimes called an ordered Bell number or a Fubini number (see the corresponding OEIS entry ). For any function , and any subset , by we denote the multiset of images of elements of . This is in contrast with the usual notation , which is the image of , i.e., the set of images of elements of . For two disjoint sets and a function , such that , we say that a function is color-preserving if for every it holds that .
Consider a hypergraph . For every vertex , let be the set of edges containing . The degree of a vertex is defined as . By we denote the maximum degree, i.e., . A hypergraph is -uniform, for , if for every . Clearly, graphs are 2-uniform hypergraphs. For a -uniform hypergraph , define
2.1 Graph labeling problems
Let be a simple, undirected graph, i.e., a 2-uniform hypergraph and consider an (unrestricted) edge labeling of . We say that distinguishes neighbors by sets (respectively, by multisets) if (respectively, ) for every pair of adjacent vertices and . Clearly, in both cases, such a labeling exists if and only if and only if the graph is nice, i.e., it does not have an isolated edge. Recall that by we denote the minimum number of labels used in an edge labeling of , distinguishing neighbors by sets.
Similarly, let be a total labeling of , i.e., a labeling of its edges and vertices. We say it distinguishes neighbors by sets (respectively, by multisets) if (respectively, ) for every pair of adjacent vertices and . Note that such a labeling exists for every graph , as it is enough to use a different label for every vertex, and one extra label for all edges.
In list variants of all four problems, edges (or edges and vertices in total labelings) are equipped with lists of possible labels, and we ask for a labeling, where the label of every edge (or edge and vertex in total labelings) belongs to the appropriate list.
It is interesting to note a very close connection between total and edge labelings distinguishing neighbors by sets. Consider a nice graph and let denote the minimum number of labels in a total labeling distinguishing neighbors by sets. First, observe that , because we can extend any edge labeling, which is neighbor distinguishing by sets, to a total labeling, by assigning to each vertex a label that appears on some edge incident to . On the other hand, consider a total labeling of , which distinguishes neighbors by sets and uses labels. By the argument analogous to the one used by Horňák and Soták , we observe that
is a proper vertex coloring of , which implies that and thus . Recall that Győri and Palmer  showed that if , then , which implies that .
Finally, if is a bipartite graph then , because we can extend its proper vertex coloring using colors to a total labeling, by assigning label 1 to every edge of . The inverse of this statement is also true: if has a total labeling , distinguishing neighbors by sets, then is bipartite. This is because the sets and form a bipartition of .
Observe that this reasoning does not show equivalence of the list variants of the problems, so it still makes sense to consider them separately.
Finally, note that if we are interested in edge/total labelings of a disconnected graph , then we can label each connected component of independently, as the distinguishing constraints are local. Thus we will focus on connected graphs.
2.2 Distinguishing neighbors via colorings of hypergraphs
Let be a hypergraph, we do not allow multiple edges. We say that a coloring distinguishes intersecting edges by sets (or, in short, is an ieds-coloring) if for every pair of distinct intersecting edges it holds that
Analogously, distinguishes intersecting edges by multisets (or is an iedm-coloring), if for every pair of distinct intersecting edges it holds that
By a list ieds-coloring (respectively, a list iedm-coloring) we mean an ieds-coloring (respectively, an iedm-coloring), in which the color of each vertex is chosen from a list which is assigned to the vertex . The lists come with a graph and are assumed to be a part of the instance.
Now, for the graph , let be its dual hypergraph, i.e., the hypergraph whose vertex set is the set of edges of , and edges of correspond to vertices of in the following way: . Observe that in such a hypergraph each vertex belongs to exactly two edges and also, if is -regular, then is -uniform and . Let be an edge labeling of a graph . We observe that distinguishes neighbors by sets (respectively, by multisets) if and only if it is an ieds-coloring (respectively, iedm-coloring) of the dual hypergraph of , see Figure 4.
Analogously to the previous case, we define the total hypergraph of , i.e., the hypergraph , whose vertices are both vertices and edges of . Every edge of is the set of all edges incident to a vertex in and the vertex itself, i.e., . Note that and, if is -regular, then is -uniform and . Now, if is a total labeling of , it distinguishes neighbors by sets (respectively, by multisets) if and only if it is an ieds-coloring (respectively, an iedm-coloring) of .
3 Upper bound
In this section we prove the main results of the paper, i.e., upper bounds on the size of lists, which guarantee the existence of an ieds-coloring or an iedm-coloring of a given -uniform hypergraph.
Let and let be a -uniform hypergraph with and , whose every vertex is equipped with a list of at least colors. Define and for every . The following hold:
For the rest of the section, let be a fixed -uniform hypergraph on vertices, such that and for some . Note that if , then there are no intersecting edges and thus the problem is trivial. We assume that sets and are linearly ordered, clearly these orderings induce a linear ordering on any subset of or . For every , denote by a position of element in the set , determined by this ordering. Suppose that every vertex of is assigned with a list of at least colors.
For a subset of , a function is called a partial ieds-coloring if for every pair of distinct, intersecting edges such that , it holds that
and for every vertex . Analogously, is called a partial iedm-coloring if for every such pair of edges we have that
and for every . A partial ieds-coloring (iedm-coloring, respectively) is complete if , note that for the condition (4) is exactly (2) and the condition (5) is equivalent to (3). Moreover, notice that conditions (4) or (5) are necessary, if we want to extend a partial ieds-coloring or a partial iedm-coloring to a complete one, see Figure 5.
Both statements in Theorem 1 can be shown in a very similar way. We will discuss the proof in a detail and point out the differences between the cases of sets and multisets. By a (partial) ied-coloring we will mean a (partial) ieds-coloring or a (partial) iedm-coloring, depending whether we want to show the statement (a) or (b).
We will construct an appropriate coloring of iteratively, ensuring that after each step the current partial coloring is a partial ied-coloring. In general, the algorithm works as follows. We fix a large and a sequence of numbers from . In each step of the algorithm we color an uncolored vertex, using the next number from , and check if the obtained partial coloring is a partial ied-coloring. If so, we proceed to the next iteration. If not, it means that the condition (4) or (5) (depending on the case of sets or multisets) is violated for some pair of edges . In this situation we say that a conflict appears on these two edges. We erase colors of some already colored vertices, to ensure that the current coloring is a partial ied-coloring. Moreover, we use an additional table to register all information about occurring conflicts. The algorithm terminates when all vertices of are colored or all numbers from the sequence are used (i.e., after iterations). In the first case the algorithm returns a complete ied-coloring. In the second one, it returns a pair , where is the table of conflicts and is the partial ied-coloring obtained after the last iteration.
For the contradiction, assume that does not have any ied-coloring, respecting the lists . This means that for every possible sequence the algorithm does not return a complete list ied-coloring of , but some pair . We will show that there is a bijection between all possible sequences and all possible pairs . Then, we will show that if is sufficiently large, then the number of all pairs is strictly smaller than , which is a number of possible sequences . This leads to a contradiction, so there is at least one sequence , for which the algorithm successfully returns a complete list ied-coloring of .
3.1 The algorithm
Let be a large integer and let be a sequence of integers from . At the beginning of the procedure all vertices of are uncolored, call this (empty) partial coloring . Also, if a list for some has more than elements, we truncate it, so that all lists are of size exactly . For , by we denote the partial ied-coloring after the -th iteration.
Recall that the vertices of are linearly ordered. For each , the -th iteration of the algorithm consists of the following steps.
- Step 1.
Find the smallest uncolored vertex in , call it .
- Step 2.
Assign the -th color from the list to and denote by the obtained partial coloring.
- Step 3.
If is a partial ied-coloring, write in and set . If is a complete ied-coloring, return it and terminate, otherwise, proceed to the next iteration.
- Step 4.
If is not a partial ied-coloring, then a conflict appeared on a pair of intersecting edges , and belongs to at least one of them, say . This is because was a partial ied-coloring and a conflict was caused by coloring . If there is more than one conflict, we can choose any of them. Let , so , and define the set
The final step differs in case of distinguishing by sets and distinguishing by multisets.
- Step 5 (variant (a): sets).
There are two types of possible conflicts that may occur, we consider them separately.
First, consider the case that . This means that all vertices from were colored in previous iterations, and . Let and . In write the quadruple , where is a color-preserving function from to (later we will specify how to choose , for now it is enough to know that it is color-preserving). After that, uncolor all vertices from , denoting the obtained coloring by , and proceed to the next iteration.
Now, consider the case that , clearly all vertices of are already colored. Observe that belongs to exactly one of the sets and ), because was a partial ieds-coloring. Without loss of generality assume that , otherwise switch the names of and .
Let be the smallest vertex of , it exists, because does not have multiple edges. Let and . In write the quadruple , where is a color-preserving function from to . After that, uncolor all vertices from , denoting the obtained coloring by , and proceed to the next iteration.
- Step 5 (variant (b): multisets).
Note that now , because was a partial iedm-coloring. Recall that . Let and . In write the triple , where is a color-preserving function from to . After that, uncolor all vertices from , denoting the obtained coloring by , and proceed to the next iteration.
Observe that after uncoloring (and possibly some additional vertices) there are no more conflicts, so is a partial ied-coloring for every . Moreover, note that the execution of the algorithm does not depend on the content of . The information stored there will only be used in the proof.
It is clear that the algorithm either returns an ied-coloring, or runs for iterations and terminates without success. In the latter situation, we say that the pair is produced from the sequence , note that this pair is uniquely determined by . Let (for input) denote the set of all possible sequences , and (for output) denote the set of all pairs that can be possibly produced from a sequence . Assume that the algorithm never returns a complete ied-coloring, which means that for every it produces some pair . This implies that .
3.2 Equicardinality of sets and
We will show that if a pair belongs to , then there is exactly one sequence that produces . By we denote the set of uncolored vertices after the -th iteration of the algorithm and by we denote the set of vertices that are colored after the -th iteration of the algorithm. Moreover, we define and . Notice that for every the sets and form a partition of , i.e., and .
First, let us prove that using the entries of only, we can reconstruct sets for all .
For every the set is uniquely determined by and .
Let be the smallest vertex in the set . In the -th iteration we assigned a color to . Depending on and, obviously, the variant of coloring we consider, we have the following possibilities:
- Case 1: (both variants (a) and (b): sets and multisets).
In this case, no conflicts arose after coloring in the -th iteration and no vertex was uncolored. Therefore
- Case 2: (variant (a): sets).
This means that there was a conflict on some intersecting edges and , such that . We know that is the -th element of the set . From the size of the domain of , we can determine the value of . Knowing , , and , we can uniquely determine the set , and thus also the edge , which is the -th element of . After the conflict occurred, we uncolored all the vertices from , so
- Case 3: (variant (a): sets).
This means that there was a conflict on some edges and , such that . We know that is the -th element of , and is the -th element of . The vertex is the smallest vertex in . After the conflict occurred, we uncolored all vertices from , so
- Case 4: (variant (b): multisets).
This case is very similar to the second one. Again we have a conflict on edges and , such that . Using , we can determine , and is given by the size of the domain of . Having , , and , we can compute , and then, using , we can find . After the conflict occurred, we uncolored all the vertices from , so ∎
Using 2, we can reconstruct sets . Now we show that using the pair that was returned by the algorithm and sets for , we can reconstruct all elements of .
For every , the partial ied-coloring and the number are uniquely determined by , , and the sets .
For every we have . Let be the smallest vertex in , this is the vertex that was colored in the -th iteration. We know that agrees with on the set . Again we consider the cases.
- Case 1: (both variants (a) and (b): sets and multisets).
This means that coloring in the -th iteration did not cause any conflict. Thus for every we have . From we get , and is the position of the color in the list .
- Case 2: (variant (a): sets).
Recall that this means that there was a conflict on some edges and , where . We determine and as in 2. To get , we need to recover colors of the vertices from , and also the number . Observe that all colors in appear on vertices of , and since is color-preserving, we can easily reconstruct the coloring on . In the same way we reconstruct the color that was given to in the -th iteration, is its position in .
- Case 3: (variant (a): sets).
This means there was a conflict on some edges and , where . We find , , and as in 2. As in the case above, we use to reconstruct colors of to get , and also the color that was assigned to to get .
- Case 4: (variant (b): multisets).
Again, we find and as in 2, and then use to reconstruct and .∎
The sets and have the same cardinality.
From 2 it follows that knowing , we can determine the sets for every . Using them, the table , and the partial ied-coloring , we can reconstruct all partial ied-colorings and the sequence , as shown in 3. This implies that every possible pair is produced by a unique sequence , which means that . Since we know that also holds, the proof is complete. ∎
3.3 Cardinality of
Now we want to estimate the number of possible pairsthat may be produced by the algorithm. Let us start with estimating the number of possible entries in the table , other than just a sign. We use the notation from the previous section. Consider an iteration , where a conflict appeared on two intersecting edges and , such that , and , and let be the coloring obtained by assigning the color to .
We will consider the cases of sets and multisets separately. For each , by (, respectively), we denote the set of all possible entries other than +, that may appear in in case of ieds-coloring (iedm-coloring, respectively). Clearly if , then there are no edges and for which holds, so there are no possible conflicts. Therefore in such a case we have . Now consider .
Variant (a): sets.
Then each entry is a quadruple or . Since , clearly we have .
Case 1: .
Note that in this case , recall , where . Equivalently, contains all edges, which do not contain , and have exactly common vertices with . There are at most edges intersecting , which implies that there are at most elements of , so
Finally, we need to estimate the number of possible color-preserving functions , recall and we want to store the information about for every . Let and , clearly . Also, define and , so we have . Observe that and . Note that and may be considered fixed, as they only depend on , , and . Fix a linear ordering of elements of , which is implied by the linear ordering of vertices of : each color is represented by the smallest vertex in this color.
We observe that if the condition (4) does not hold, every color from must appear on some vertex of . On the other hand, colors from may appear on vertices in , but do not have to. Let and . Now is a surjective function from to and can be chosen in at most
ways, where denotes the number of surjective functions from an -element set to a -element set. We observe that
where the last inequality follows from (1). Summing up, we obtain that the number of possible functions is at most
Case 2: .
If , then , so and thus clearly . The bound on the number of functions is obtained in a way analogous to the previous case. Again, we are interested in bounding the number of surjective functions from to , where is the number of colors from that appear in , and . We observe that this number is bounded by (6).
Summing up, we conclude that
Variant (b): multisets.
This variant is significantly simpler. The only possible entry in is a triple . Just as in the Case 1 in the previous variant, we have
By condition (5), can be assumed to be a bijection, so it can be chosen in ways. So, summing up, we obtain the following bound:
The rest of the proof is exactly the same in both variants: distingushing by sets and multisets. For every , by let us denote , if we are interested in finding an ieds-coloring, or , if we are interested in finding an iedm-coloring. Define .
Now, let us bound the number of all possible tables that could be produced by the algorithm, denote it by . By denote a number of symbols in . For every let be the number of appearances of the elements of in ; if , then clearly . Notice that . Denote by
the number of partitions of an -element set into the subsets of cardinalities .
The number of all possible tables is bounded from above by
For every , the total number of appearances of elements of in is . It implies that for fixed , the number of ways to fill the table is at most
Observe that if an element of appears in , it means that in the -th iteration we uncolored vertices that were colored in previous iterations. Thus to each occurrence of an element of in we can assign iterations when we wrote into , and each symbol is assigned at most once. Let be the number of iterations when a conflict occurred, or when we colored a vertex that was uncolored later. There are iterations when a conflict occured, and iterations when we colored a vertex which was uncolored later, so
Clearly is at most , which is the total number of iterations. On the other hand, after steps the algorithm returns a partial ied-coloring, where at most vertices are colored. Therefore . ∎
We will need the following technical lemma shown in [15, Lemma 2.7].
Lemma 6 (Bosek, Czerwiński, Grytczuk, Rz. ).
If are non-negative integers such that and , then
Now we are ready to prove the final lemma.
where and for Then there exists such that for every the number of elements of the set is strictly smaller than .