# Some results relating Kolmogorov complexity and entropy of amenable group actions

It was proved by Brudno that entropy and Kolmgorov complexity for dynamical systems are tightly related. We generalize his results to the case of arbitrary computable amenable group.

## Authors

• 2 publications
• ### On the relation between topological entropy and restoration entropy

In the context of state estimation under communication constraints, seve...
11/28/2018 ∙ by Christoph Kawan, et al. ∙ 0

• ### A topological dynamical system with two different positive sofic entropies

A sofic approximation to a countable group is a sequence of partial acti...
11/19/2019 ∙ by Dylan Airey, et al. ∙ 0

• ### Undecidability and Irreducibility Conditions for Open-Ended Evolution and Emergence

Is undecidability a requirement for open-ended evolution (OEE)? Using me...
06/06/2016 ∙ by Santiago Hernández-Orozco, et al. ∙ 0

• ### Generic Transformations Have Zero Lower Slow Entropy and Infinite Upper Slow Entropy

The notion of slow entropy, both upper and lower slow entropy, were defi...
06/27/2020 ∙ by Terry Adams, et al. ∙ 0

• ### Genericity and Rigidity for Slow Entropy Transformations

The notion of slow entropy, both upper and lower slow entropy, was defin...
06/27/2020 ∙ by Terry Adams, et al. ∙ 0

• ### Complexity-based permutation entropies: from deterministic time series to white noise

This is a paper in the intersection of time series analysis and complexi...
03/05/2021 ∙ by J. M. Amigó, et al. ∙ 0

• ### Preservation of normality by non-oblivious group selection

We give two different proofs of the fact that non-oblivious selection vi...
05/03/2019 ∙ by Olivier Carton, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

In the works [Br74], [Br82] A. Brudno have showed that the entropy of an action is intimately connected with the Kolmogorov complexity of its points. Namely, he proved that the mean Kolmogorov complexity of any poin in the subshift is bounded from above by the topological entropy of the shift, and that there is a point whose mean complexity equals the entropy of the subshift. On the measure-theoretic hand, he proved that, for an ergodic invariant measure on a shift-action, almost every point has the mean complexity equal to the Kolmogorov-Sinai entropy of the measure.

Later on, the topological part of these results were generalized to the case of a action by Simpson in [Si15]. Alexander Shen brought my attention to the questions in the latter work concerning generalization of Brudno’s results to the case of an arbitrary amenable group.

In my 2013 diploma work [A13] I’ve managed to answer some questions from Simpson’s paper. It is natural to require the group to be computable. Elements of the Følner sequene with high complexity can complicate our matters. So we explicitly require the sequence to satisfy the “modesty” assumption: complexity of a Følner set is negligible relative to its size. We note that there are such Følner sequences in any computable amenable group. Moreover, theorem 6 presents a very natural sufficient geometric criterion for a Følner sequence of a finitely-generated group to be modest. If elements of a Følner sequence are edge-connected on the Cayley graph and all contain the group identity element, then the sequence is modest. The following theorem consitutes the main result of my master’s thesis [A13]. That work is written in Russian and have never been published, so I’ve decided to incorporate its results into this paper. Let be an amenable group with a Følner sequence . Let be a finite set. For any we denote

 ¯¯¯¯¯¯¯¯ACF(x)=limsupi→∞C(prFi(x))|Fi|

the upper asympotic complexity of relative to , and

 AC––––F(x)=liminfi→∞C(prFi(x))|Fi|

the upper asympotic complexity of relative to . Here stands for the Kolmogorov complexity function.

###### Theorem 1.

Let be a subshift over a computable amenable group. Let be a modest Følner sequence. For every holds

 ¯¯¯¯¯¯¯¯ACF(x)≤htopG(X).

If is also tempered, then there is such a point that

 ¯¯¯¯¯¯¯¯ACF(y)=AC––––F(y)=htopG(X).

If in addition has continuum cardinality, then there is a continuum of such points .

In proposition 6 from section 7 we handle the upper bound for the complexity in terms of the topological entropy. Proposition 5 from section 7 proves that there is a poin with the complexity not smaller than the topological entropy. The last assertion of the theorem above is due to the fact that if the topological entropy is zero, then every poin will do; if it is positive, then there is a measure of maximal entropy, and almost every point satisfies the requirement. Since the entropy is positive, there is a continuum of these points.

Afterward, Moriakov in his papers [Mo15a] and [Mo15b] independently considered the generalizations of Brudno’s results to the case of amenable groups having so-called computable Følner monotiling. The latter means that the group has a special Følner sequence each of whose elemements can tile the whole group, and that this tiling can be generated by a computer program. This requirement is known to be rather restrictive.

Motivated by Moriakov’s work, I devised a proof for the general case of measure-entropy Brudno’s theorem:

###### Theorem 2.

Let be a computable amenable group. Let be a modest tempered Følner sequence Let be an ergodic inariant measure on . Then for -a.e. we have

 ¯¯¯¯¯¯¯¯ACF(x)=AC––––F(x)=hG(AG,μ).

Proof naturally splits up into lower and upper bounds. The lower bound for the complexity is proved in the proposition 2 from section 4. The upper bound constitutes proposition 4 from section 5. The weaker “mean” form of the latter proposition was used by Bernshteyn in [Be16].

The results of this paper have been announced in [A18].

Acknowledgements. Research is supported by the Russian Science Foundation grant No14-21-00035. I would like to thank Alexander Shen for suggestion to work on generalizations of Brudno’s results, and for helpful discussions. Discussions with Pavel Galashin were instrumental in proving theorem 6. I’d like to thank Sergey Kryzhevich for his comments on my diploma work [A13]; results from that work constitute a substantial part of this paper.

## 2 Preliminaries

### 2.1 Computability and Kolmogorov complexity

In the sequel we may assume without loss of generality that our notion of computability is augmented with a fixed oracle. Denote the set of all finite binary strings (the empty one included). For we denote the length of . Let be a computable function (not necessarily everywhere defined). For we denote the smallest such that ( if there are no such ’s). By the Kolmogorov-Solomonoff theorem, there is a computable function such that for any other computable there is a constant such that for any holds

 Cf(x)

We fix any such “optimal” . The Kologorov complexity of a string is defined as and denoted by . It is easy to see that for any natural there are at most words satusfying the bound . It is worth to note that we can define the Kolmogorov complexity as a function on any class of finite constructible combinatorial objects. One important example: there is a constant K such that for any finite set and any holds

 C(x)≤C(T)+log|T|+K.

Another example is complexity of tuples. For a fixed natural there is such that for any tuple holds

 C(x1,…,xk)≤C(x1)+…+C(xk)+2log(C(x1))+…+2log(C(xk−1))+K.

We refer the reader to books [LV97] and [SUV] for the details.

### 2.2 Computable amenable groups

Let be a countable group. Let be a sequence of finite subsets of . The sequence is called a Følner sequence for if for every holds

 limi→∞|gFi∖Fi||Fi|=0.

A countable group is called an amenable group if it has a Følner sequence.

Let be a countable group. We will say that is a computable group if the set of its elements is the set of natural numbers, and the composition function is computable. We may asssume also that is the identity element. We sometimes will use the order inherited from the set of natural numbers for compuable groups. Note that a finitely-generated group is computable whenever the word problem for the group is decidable. A computable amenable group is simply a computable group which is amenable. As was mentioned earlier, we can define the Kolmogorov complexity of a finite subset of . We will say that a Følner sequence of a computable amenable group is modest if the following bound holds:

 C(Fi)=o(|Fi|).

This definition was given in my work [A12], the following was also proved in that work.

###### Theorem 3.

Every computable amenable group has a modest Følner sequence.

###### Sketch of proof.

Let us fix an enumeration of finite subset of . For let be the first subset such that and that for every (the order is the order of natural numbers). It is easy to see that this sequence is Følner and modest. ∎

In section 3 we will introduce a nice geometric criterion for a Følner sequence to be modest, which works in a lot of situations.

### 2.3 Actions and ergodic theory

We refer the reader to books [Gl03] and [EW11] for the introduction to ergodic theory. Results amenable to the ergodic theory of amenable groups could be found in [Oll85], [OW87],[L01], [KL16].

Let be a countable group, let be a finite alphabet. The shift-action of group on the space is defined by

 (gx)(h)=x(hg),

for every and . We endow the space with the product topology. It is easy to note that the action defined is an action by homeomorphisms. A map is said to be equivariant if for every and . A subshift is any closed invariant subset of . We can also mean by it the corresponding restriction of the action.

We denote the set of all partial maps from to with finite domains. We denote the domain of for . We denote the word

 (t(g1),…,t(g|supp(t)|)),

where , and ’s are listed in the increasing order.

Let be a countable group. By an action of on a standard probabity space we will always mean a measurable and measure-preserving action. Every invariant measure on defines an action of on .

Suppose a countable group

acts on two standard probability spaces

and . A measure-preserving map is said to be a factor map if for -a.e. . It is an isomorphism if it is one-to-one on a set of full measure.

Suppose a countable group acts on a standard probability space . Let be a measurable map to a finite set . We can extend this map to by the equality

 (π(x))(g)=¯π(gx).

Let be a standard Borel space or standard probability space. A partition of is a finite or countable collection of subsets of whose union is the whole set. For two partitions and we denote the partition .

Consider an action of a countable group on a standard probability space . For a partition of and an element we denote the partition . Partition is said to be generating if the smallest -subalgebra containing for each is (modulo sets of mesure ) the algebra of all measurable subsets of . The following condition is well-known to be equivalent: there is a subset of full measure such that for every pair with there is an element such that and belong to different pieces of .

A probability vector is a countable or finite sequence of non-negative numbers whose sum is

. Let be a probability vector. It’s Shannon entropy is defined as

 −∑ipilog(pi),

with the convention , and denoted by . Any partition of a probability space defines a probability vector. For a partition we denote the entropy of the corresponding probability vector.

Let a countable amenable group act on a standard probability space , let be a partition of finite Shannon entropy. Let be a Følner sequence for . We denote

 hG(α,X,μ)=limi→∞H(αFi)|Fi|

It is well known that this limit does not depend on the choice of the Følner sequence. The Kolmogorov-Sinai entropy of the action is defined by the formula

 supH(α)<∞hG(α,X,μ).

Consider the shift-action of a countable group on . Preimages of elements of under the map from to constitute a partition. This partition is called a canonical alphabet generating partition.

Let a countable group act on a standard probability space . Suppose is a map from to a finite set . Let be the corresponding partition of (namely, the partition consisting of preimages under of elements in ). Suppose is a generating partition. It is well known that the extended map is a bijection on a subset of full measure. We endow now with the push-forward measure . It follows that is an isomorphism of -actions. Let be an inverse of , we can assume that this map is Borel. So is an isomorphism from to . This map is an isomorphism of actions also. Alltogether we constructed a symbolic representation, corresponding to partition , of the action of on .

An action of a countable group on a standard probability space is called essentially free if it is free on the orbit of almost every point.

###### Theorem 4 (Seward and Tucker-Drob).

Consider an essentially free action of a countable group on a standard probability space . If , then for every there is a symbolic representation of this action by the shift-action on , such that the canonical alphabet generating partition has entropy smaller than .

Strictly speaking, the theorem above is a combination of corollary 2.7 from [SeTD16] which states that the so-called Rokhlin entropy is equal to the Kolmogorov-Sinai entropy for free actions of amenable group, and the main result of [Se14] asserting that the statement of the theorem above holds after substituting the Kolmogorov-Sinai entropy with the Rokhlin entropy.

### 2.4 Topological entropy

For a compact metric space we denote the size of a mininmal -spanning net.

Let be a compact metrizable space. Let be a countable amenable group acting by homeomorphisms on . Let us fix a a compatible metric on . Suppose is a finite subset of ; we denote

 lS(x,y)=supg∈Sl(gx,gy),

for . Let us fix a Følner sequence for . Topological entropy of the action is defined by

 supε>0limi→∞logNε(X,lFi)|Fi|

and denoted as . It is well known that this does not depend on the choice of the metric and Følner sequence.

For a subshift there is the following equivalent definition

 htopG(X)=limi→∞log|prFi(X)||Fi|.

We will need the following implication of the variational principle.

###### Theorem 5.

Let be a finite set and be an amenable group, let be a subshift. There is such an invariant Borel probability measure on that

 hG(X,μ)=htopG(X).
###### Proof.

This follows from the variational principle together with the upper-semicontinuity of measure entropy as a function on the set of invariant measures for an expansive dynamical system. See chapter 5 from [Oll85]. ∎

### 2.5 Asymptotic complexity

Let be a finite alphabet. Consider the shift-action of a computable amenable group on . Let be a Følner sequence. The upper asymptotic complexity of a point relative to is defined as

 limsupi→∞C(prFi(x))|Fi|

and denoted by . The lower asymptotic complexity of a point relative to is defined as

 liminfi→∞C(prFi(x))|Fi|

and denoted by .

It is important to note that, if is a modest Følner sequence, then

 ¯¯¯¯¯¯¯¯ACF(x)=limsupi→∞C(cont(prFi(x)))|Fi|

for every ; analogous equality holds for the lower asymptotic complexity.

## 3 Reflections concerning modest Følner sequences

The following proposition will be needed to apply the Borel-Cantelli lemma in the proof of proposition 2.

###### Proposition 1.

Let be a modest Følner sequence in a computable amenable group . If for every pair with , then for every the following serie converges:

 ∑i∈N2−ε|Fi|.
###### Proof.

Throwing away a finite initial segment of the Følner sequence, we may assume that for every . We rearrange the sum:

 ∑k∈N2−εk|{i:|Fi|=k}|.

For each with we have ; so for each we have . This means that the sum is bounded by the geometric serie:

 ∑k∈N2−εk/2.

###### Theorem 6.

Suppose is a computable amenable group which is finitely generated. Let us fix a finite symmetric generating set . This defines the structure of the Cayley graph on . Let be a Følner sequence. If each is an edge-connected subset of the Cayley graph and contains the group identity, then is a modest Følner sequence.

###### Proof.

Algorithm 1 takes a finite connected subset which contains and reversibly encodes it as a binary string consisting of ’s and ’s. This algorithm is based on the standard depth-first search algorithm for graphs(see [CLRS] chapter 22.3).

I would like to note that the output string of algorithm 1 uniquelly determines the input set. Algorithm 2 decodes the string. I note that stands for the ball of radius in Cayley graph around the group identity. If we apply the encoding to an element of the Følner sequence, we get the binary string of length ; this string contains a negligible amount of ’s; hence, by lemma 1, we have .

## 4 Measure entropy; lower bound for complexity

Let be a partition of a standard probability space . For denote the element of that contains . We will need the following generalization of the Shannon-McMillan-Breiman theorem due to Lindenstrauss [L01]:

###### Theorem 7.

Consider an ergodic action of a countable amenable group on a standard probability space . Let be a partition of finite Shannon enropy. Let be a tempered Følner sequence. For -a.e. holds

 limi→∞−logμ(αFi(x))|Fi|=hG(α,X,μ)

.

###### Proposition 2.

Let be an ergodic invariant measure for the shift-action of a computable amenable group on . If is a modest tempered Følner sequence, then for -a.e. we have

 AC––––F(x)≥hG(AG,μ).
###### Proof.

Denote . Without loss of generality we may assume that the Følner sequence does not have repeating elements. Let be a canonical alphabet generating partition. Let . By the theorem above, there is a subset with and a number such that for each and we have

 logμ(αFi(x))≤−(h+ε)|Fi|.

In order to prove the proposition it is enough to show that for -a.e. we have

 AC––––F(x)≥h−2ε.

For each , denote the subset of all such that . By Borel-Cantelli lemma, it suffices to show that

 ∑i>Nμ(Si)<+∞.

We proceed by showing that this is the case. For each there are at most such that . This implies that

 μ(Si)≤2(h−2ε)|Fi|⋅2−(h−ε)|Fi|=2−ε|Fi|.

So the desired convergence holds due to proposition 1.

## 5 Measure entropy; upper bound for complexity

Let be an ergodic measure on ; we will show that (under some additional requirements) the upper asymptotic complexity of -a.e. point is bounded from above by the Kolmogorov-Sinai entropy. Here’s the general outline of the proof. We show that the asympotic complexity is essentially monotone relative to factor maps. This is done first for cellular maps(which is immediate); then we propagate this to any factor-map by means of an approximation argument. We use theorem 4 of Seward and Tucker-Drob to show that the desired bound on the complexity holds if the action is essentially free. We relieve the freeness assumption by taking a product with a Bernoulli action of small entropy.

We will use the following ergodic theorem of Lindenstrauss([L01]:

###### Theorem 8.

Let be a countable amenable group acting ergodically on a standard probability space . Let be an function on . If is a tempered Følner sequence, then

 limi→∞1|Fi|∑g∈Fif(gx)=E(X,μ)f.

Let be a finite alphabet. For a word we denote the probability vector for occurence rates of letters from in .

The following is lemma 146 from book [SUV].

###### Lemma 1.

For every holds

 C(w)≤|w|(H(p(w))+o|w|(1)).
###### Lemma 2.

Consider the shift-action of a computable amenable group on for a finite set . Let be an ergodic invariant measure. Let be a canonical alphabet generating partition. If is a tempered Følner sequence, then

 limsupi→∞C(cont(prFi(x)))|Fi|≤H(α),

for -a.e. . If in addition is modest, then

 ¯¯¯¯¯¯¯¯ACF(x)≤H(α),

for -a.e. .

###### Proof.

The first assertion is a consequence of Lindenstrauss’s ergodic theorem together with the lemma above. The second one follows easily from the observation that for modest Følner sequences and for every holds

 C(prFi(x))=C(cont(prFi(x)))+o(|Fi|).

Let and be two words in of the same length. The Hamming distance between and is defined as

 |i:w′(i)≠w′′(i)||w′|,

and denoted by .

The lemma shows two words have close complexity rates given that these words are close in the Hamming metric.

###### Lemma 3.

For two words , in of the same length the following holds:

 |C(w′)−C(w′′)|≤n(H(DH(w′,w′′),1−DH(w′,w′′))+DH(w′,w′′)log|A|+on(1)).
###### Proof.

In order to recover from it is enough to provide the word from that will encode the places where and differ, and the word from that will encode the letters to substitude. The first word contains exactly of ’s. So its complexity is bounded by

 nH(DH(w′,w′′),1−DH(w′,w′′))+on(n).

The second has complexity bounded by

 nDH(w′,w′′)log|A|+O(1).

The statement now follows from standard properties of the Kolmogorov complexity. ∎

For two elements with we define the Hamming distance between them by

 |{g∈supp(t1),t1(g)≠t2(g)}||supp(t1)|

and denoted as . Let be a Følner sequence. For two elements we define the asymptotic Hamming distance between them relative to by the formula

 limsupi→∞DH(prFi(x1),prFi(x2))

and denoted by .

###### Lemma 4.

Let and be finite sets. Suppose and are two equivariant maps from to . Suppose is an ergodic invariant measure on . Let be the set of all such that , where stands for the identity element in group . Let be a tempered Følner sequence. For -a.e. holds

 dHF(π′(y),π′′(y))=μ(W).
###### Proof.

It is easy to note that is equivalent to , which in turn is equivalent to . Now the statement follows from Lindenstrauss’s ergodic theorem (theorem 8). ∎

###### Lemma 5.

In the setting of the previous lemma let be a modest Følner sequence. For -a.e. we have

 |¯¯¯¯¯¯¯¯ACF(π′(y))−¯¯¯¯¯¯¯¯ACF(π′′(y))|≤H(μ(W),1−μ(W))+μ(W)log|A|.
###### Proof.

This statement is a consequence of the previous lemma together with lemma 3. ∎

###### Lemma 6.

If is a cellular map, then for every and every Følner sequence holds

 ¯¯¯¯¯¯¯¯ACF(π(y))≤¯¯¯¯¯¯¯¯ACF(y).
###### Proof.

Suppose is the memory set of cellular map . It is easy to notice that could be uniformly computably recovered from and . Now the statements follows from the definition of a Følner sequence. ∎

###### Proposition 3.

Let be a measurable equivariant map. Let be an ergodic invariant measure on . For -a.e. holds the following bound:

 ¯¯¯¯¯¯¯¯ACF(π(y))≤¯¯¯¯¯¯¯¯ACF(y).
###### Proof.

Take arbitrary . We aim to prove that for -a.e. . There is such an that for every non-negative we have

 H(ε′′,1−ε′′)+ε′′log|A|≤ε.

Let be a cellular map from to such that for the set of all the points for which , we have . Lemma 6 implies that for every . Lemma 5 implies that for -a.e. . It follows that for -a.e. . ∎

###### Proposition 4.

Let be a finite set. Consider the shift-action of a computable amenable group on . Let be an invariant ergodic measure on . Let be a modest and tempered Følner sequence. For -a.e. holds

 ¯¯¯¯¯¯¯¯ACF(x)≤hG(AG,μ)
###### Proof.

Denote . Fix any . We will prove that for -a.e. holds

 ¯¯¯¯¯¯¯¯ACF(x)≤h+ε.

Let be a Bernoulli action of entropy smaller than . Consider the product-action . This action is ergodic and essentially free. Its entropy is smaller than . By theorem 4, it has a symbolic representation such that for the canonical alphabet generating partition holds . So we have a factor-map from to . By lemma 2, we have for -a.e. . By proposition 3 we have that for -a.e. holds . This implies that for -a.e. . ∎

## 6 Topological entropy; lower bound for complexity

###### Proposition 5.

Let be a finite alphabet and let be a computable amenable group. Let be a subshift. If is a modest Følner sequence in , then there is a point such that

 AC––––F(x)≥htopG(X).
###### Proof.

By theorem 5, there is a measure such that . There is a point with

 AC––––F(x)≥hG(X,μ)

by proposition 2. ∎

## 7 Topological entropy; upper bound for complexity

###### Lemma 7.

Let be a computable amenable group.Let be a Følner sequence. For every rational there is a finite sequence and a number such that the following holds. There is a program wich receives a finite subset . If for some , then it outputs the sequence of subsets satisfying the assertions below; otherwise its behaviour is undefined.

1. ,

2. ,

3. .

###### Proof.

If we drop the requiremnt for to be generated by a program, the statement becomes a variant of the Ornstein-Weiss covering lemma (see [OW87] and [KL16] lemma 9.23). The program will just search for the output among finitely many variants. ∎

###### Proposition 6.

Let be a computable amenable group. Let be a finite alphabet. Suppose is a subshift. Let be a modest Følner sequence for group . For every holds

 ¯¯¯¯¯¯¯¯ACF(X)≤htopG(X).
###### Proof.

Denote . Take arbitrary rational . It suffices to prove that for every holds

 ¯¯¯¯¯¯¯¯ACF(x)≤(1+ε)(h+ε)+εlog|A|.

Throwing away a finite initial part of , we may assume that

 log∣∣prFi(X)∣∣<(h+ε)|Fi|

We apply the previous lemma. This lemma gives us a collection . We also record the finite subsets for the future use. Now we will consruct a program that will take a finite subset of , and if for some , it will output the set of all satisfying for every and (note that is equivalent to ). It is easy to see that for we have

 log|Q(Fi)|≤((1+ε)(h+ε)+εlog|A|)|Fi|.

Indeed, denote

 T′=k⋃s=1FjsKs(Fi)

it follows from the previous lemma that

 log|prT′(Q(Fi))|≤k∑s=1∑m∈Ki(Fs)log∣∣prFjsm(Q(Fi))∣∣≤(1+ε)(h+ε)|Fi|.

Bound (7) follows since

 log|Q(Fi)|≤log|prT′Q(Fi)|+|Fi∖T′|log|A|≤log|prT′Q(Fi)|+ε|Fi|log|A|.

Note that for we have . We also have . The properties of the Kolmogorov complexity imply that for every and for , holds

 C(y)≤((1+ε)(h+ε)+εlog|A|)|Fi|+oi(|Fi|).

Hence, (7) holds for every . ∎