## 1 Introduction

One of the tools to study the internal structure of groups
is the normalizer.
For two groups and , which are contained in a common overgroup , we
call the *normalizer of in *,
denoted ,
the subgroup of consisting of those elements that leave invariant under
conjugation.

We only consider finite sets, finite groups, and permutation groups acting on
finite sets.
We assume permutation groups to always be given by generating sets and say that
a problem for permutation groups can be solved in *polynomial time*,
if there exists an algorithm which, given permutation groups of degree ,
solves it in time bounded polynomially in and in the sizes of the given
generating sets.
While many problems for permutation groups can be solved efficiently both in
theory and in practice,
no polynomial time algorithm to compute normalizers of permutation groups is
known.

A transitive permutation group acting on a set is called
*primitive* if there exists no non-trivial -invariant partition of
.
Primitive groups have a rich and well-understood structure.
Hence many algorithms use the natural recursion from general permutation groups
to transitive and in turn to primitive ones.
For two permutation groups computing the normalizer of
in in general is done by searching for the normalizer of in the
symmetric group and simultaneously computing the intersection
with .
We focus on computing the normalizer of a primitive group in .
Being able to compute normalizers for primitive groups efficiently may lead to
improved algorithms for more general situations.

Our results build substantially on the O’Nan-Scott classification of primitive groups, see [liebeck-praeger-saxl:onan-scott], and on the classification of finite simple groups (CFSG).

Recall that the *socle* of a group , denoted , is the subgroup
generated by all minimal normal subgroups of .
Our theoretical main result is the following theorem.

###### Theorem 1.1 ([siccha:phd-thesis, Theorem 9.1]).

Let a primitive group with non-regular socle^{1}^{1}1This excludes groups of affine and of twisted wreath type.
be given.
Then we can compute in polynomial time.

As is often the case in computational group theory, ideas from theoretical algorithms can be employed in practical algorithms and vice versa. While the algorithms in [siccha:phd-thesis] are primarily theoretical ones, we also provide probabilistic nearly-linear time versions where possible. The author is developing the GAP package NormalizersOfPrimitiveGroups, hosted at

https://github.com/ssiccha/NormalizersOfPrimitiveGroups^{2}^{2}2may move to https://github.com/gap-packages/NormalizersOfPrimitiveGroups,

with the aim to implement practical versions of the algorithms developed in [siccha:phd-thesis]. Until now, algorithms concerning permutation morphisms and primitive groups of type PA are implemented. First experiments indicate that already for moderate degrees these outperform the GAP built-in algorithm Normalizer by several orders of magnitude, see Table 1.

Since no polynomial time solutions are known for the normalizer problem, the
generic practical algorithms resolve to backtracking over the involved groups
in one way or another.
The fundamental framework of modern backtrack algorithms for permutation groups
is Leon’s partition backtrack algorithm [leon],
which generalizes previous backtrack approaches
[butler:backtrack, butler:normalizers, holt:normalizers, sims:backtrack]
and generalizes ideas of *nauty* [mckay-piperno:nauty-traces] to
the permutation group setting.
Partition backtrack is implemented in GAP
[gap]
and Magma
[magma].
Recently, the partition backtrack approach was generalized
to a “graph backtrack” framework
[JPWW].

Theißen developed a normalizer algorithm which uses orbital graphs to prune the backtrack search [theissen]. Chang is currently developing specialized algorithms for highly intransitive permutation groups, her PhD thesis should appear shortly. It is to expect that the work in [JPWW] can also be extended to normalizer problems. Hulpke also implemented normalizer algorithms in [hulpke:normalizers] using group automorphisms and the GAP function NormalizerViaRadical based on [glasby-slattery:normalizers]

## 2 Strategy

We describe the strategy of the theoretical algorithm behind Theorem 1.1. Comments regarding the implementation of its building blocks are given at the end of each following section.

In this section let be a primitive group with non-regular socle . The normalizer of in plays a central role in our algorithm, in this section we denote it by . Observe that to compute it suffices to compute since the former is contained in .

The socle is isomorphic to for some finite non-abelian simple
group and some positive integer .
The group is isomorphic to a subgroup of the wreath product
,
see Section 3 for a definition of wreath products.
By the O’Nan-Scott Theorem
the respective isomorphism extends to an embedding^{3}^{3}3For twisted wreath type the situation is slightly more complicated.
of the normalizer into .
Furthermore is of the order .
Hence the index of in , and thus also the search-space of the
normalizer computation , is tiny in comparison to the index of in
.

Our approach can be divided into two phases.
First we compute ,
this is by far the most labor intensive part.
To this end we compute a sufficiently well-behaved conjugate
of , such that we can exhibit the wreath structure mentioned above.
In [siccha:phd-thesis] we make this more precise and define a
*weak canonical form* for primitive groups.
Using that conjugate and the O’Nan-Scott Theorem we can write down generators
for .
In the second phase,
we compute a reduction homomorphism
with .
After this logarithmic reduction,
we use
Daniel Wiebking’s simply exponential time
algorithm
[wiebking:normalizers-simply-exponential, wiebking:normalizers-simply-exponential-siam],
which is based on the canonization framework [schweitzer-wiebking:canonisation-framework],
to compute .
Note that the running time of a simply exponential time algorithm
called on a problem of size is
and thus is bounded by for some constant .
Then we use Babai’s famous quasipolynomial time algorithm for graph-isomorphism
[babai:graphiso-quasipolynomial, babai:graphiso-quasipolynomial-acm]
to compute the group intersection
.
Notice that since we perform these algorithms on at most
points they run in time polynomial in .
The homomorphism is constructed in such a way,
that computing the preimage of the above normalizer
yields .
Recall that is equal to .

In our implementation we do not use the algorithms by Wiebking and Babai since these are purely theoretical. Instead we use the partition backtrack implemented in GAP.

## 3 The O’Nan-Scott Theorem

The goal of this and the next section is to illustrate how we use the O’Nan-Scott Theorem to prove the following theorem. In this article we limit ourselves to groups of type PA, which we define shortly.

###### Theorem 3.1 ([siccha:phd-thesis, Theorem 8.1]).

Let a primitive group with non-abelian socle be given. Then we can compute in polynomial time.

The O’Nan-Scott Theorem classifies how the socles of primitive groups can act, classifies the normalizers of the socles, and determines criteria to decide which subgroups of these normalizers act primitively. We follow the division of primitive groups into eight O’Nan-Scott types as it was suggested by László G. Kovács and first defined by Cheryl Praeger in

[praeger:onan-scott-eight-types]. In this section we define the types AS and PA and recall some of their basic properties. In particular we describe the normalizer of the socle for groups of type PA and how to construct the normalizer of the socle, if the group is given in a sufficiently well-behaved form.The version of the O’Nan-Scott Theorem we use, for a proof see [liebeck-praeger-saxl:onan-scott], is:

###### Theorem 3.2.

Let be a primitive group on a set . Then is a group of type HA, AS, PA, HS, HC, SD, CD, or TW.

The abbreviation AS stands for Almost Simple.
A group is called *almost simple* if it contains a non-abelian simple
group and can be embedded into the automorphism group of said simple group.
A primitive group is of *AS type* if its socle is a
non-regular non-abelian simple group.

The abbreviation PA stands for Product Action. The groups of AS type form the building blocks for the groups of PA type. To define this type, we shortly recall the notion of wreath products and their product action.

The *wreath product* of two permutation groups
and
is denoted by and defined as the semidirect product
where acts per conjugation on by permuting its
components.
We identify and with the corresponding subgroups of
and call them the *base group* and the *top group*, respectively.

For two permutation groups and
the *product action* of the wreath product
on the set of tuples is given by letting
the base group act component-wise on
and letting the top group act by permuting the components of .

###### Definition 3.3.

Let be a primitive group.
We say that is of *type PA* if there exist an
and a primitive group of type AS
such that is
permutation isomorphic to
a group
with

in product action on .

The product action wreath products and are examples for primitive groups of type PA.

Let and be as in Definition 3.3. We sketch how to construct the normalizer of the socle of . Let . Since is given acting in product action we can read off and thus compute . By [dixon-mortimer:permutation-groups, Lemma 4.5A] we know that the normalizer of in is . By recent work of Luks and Miyazaki we can compute the normalizer of , in polynomial time [luks-miyazaki:normalisers-poly, Corollary 3.24]. More precisely this approach yields the following corollary:

###### Corollary 3.4.

Let be a primitive group of type PA with socle in component-wise action on . Then can be computed in polynomial time.

In the practical implementation we use the GAP built-in algorithm to compute the normalizer of in . Our long-term goal is to use the constructive recognition provided by the recog package [recog1.3.2]. Computing the normalizer of in is then only a matter of iterating through representatives for the outer automorphisms of .

## 4 Permutation morphisms

In general a group of PA type might be given on an arbitrary set and needs only be permutation isomorphic to a group in product action. In this section we discuss how to construct such a permutation isomorphism:

###### Lemma 4.1.

Let be a primitive group of type PA. Then we can compute a non-abelian simple group , a positive integer , and a permutation isomorphism from to a permutation group such that the socle of is in component-wise action on .

To this end we present the notion of *permutation morphisms*
developed in [siccha:phd-thesis].
They arise from permutation isomorphisms by simply dropping the condition that
the domain map and the group homomorphism be bijections.
We illustrate how to use them to prove Lemma 4.1.

### 4.2 Basic Definitions

For two maps and we denote by the product map . For a right-action of a group and , we also denote by .

###### Definition 4.3.

Let and be permutation groups on sets and ,
respectively,
let
be a map,
and let
be a group homomorphism.
Furthermore let and be the natural actions of and on
and , respectively.
We call the pair a
*permutation morphism from to *
if the following diagram commutes:

, |

that is if
holds for all , .
We call
the *group homomorphism of *
and the
*domain map of *.

It is immediate from the definition,
that the component-wise composition of two permutation morphisms again
yields a permutation morphism.
In particular we define the *category of permutation groups*
as the category with all permutation groups as objects,
all permutation morphisms as morphisms,
and the component-wise composition as the composition of permutation
morphisms.
We rely on this categorical perspective in many of our proofs.

We denote a permutation morphism from a permutation group to a permutation group by . When encountering this notation keep in mind that itself is not a map but a pair of a domain map and a group homomorphism. We use capital letters for permutation morphisms.

It turns out that a permutation morphism is a mono-, epi-, or isomorphism in the categorical sense if and only if both its domain map and group homomorphism are injective, surjective, or bijective, respectively.

For a permutation group we call a map
*compatible with * if there exists a group
homomorphism such that is a permutation morphism.
We say that a partition of is -invariant if
for all and we have .

###### Lemma 4.4 ([siccha:phd-thesis, Lemma 4.2.10]).

Let be a permutation group and a map. Then is compatible with if and only if the partition of into the non-empty fibers is -invariant.

If is transitive, then the -invariant partitions of are precisely the block systems of . Hence for a given blocksystem we can define a compatible map by sending each point to the block it is contained in.

Let be a permutation group
and a surjective map compatible with .
Then there exist a unique group
and a unique group homomorphism
such that is a permutation epimorphism,
see [siccha:phd-thesis, Corollary 4.2.7].
We call the *permutation epimorphism*
and the
*group epimorphism of induced by *.

###### Example 4.5.

Let , , , and . Further consider the set , the map and the following geometric arrangement of the points :

Observe that acts on by permuting the points horizontally, while acts on by permuting the points vertically. The map projects vertically or “to the top”. Notice how the fibers of correspond to a block-system of . We determine the group epimorphism of induced by . By definition is the permutation which makes the following square commute:

Take . We have , , and . Hence . Correspondingly we get .

### 4.6 Products of permutation morphisms

For two permutation groups and
we define the *product of the permutation groups and *
as the permutation group
given by
in component-wise action
on .
Correspondingly,
for an additional permutation group
and two permutation morphisms and from
to and , respectively,
we define the *product permutation morphism *
as .
Iteratively, we define the product of several permutation groups or
permutation morphisms.

To prove Lemma 4.1 it suffices to be able to compute the following: given the socle of a PA type group compute a non-abelian simple group and permutation epimorphisms, think projections, such that the product morphism is an isomorphism. Since every surjective map compatible with induces a unique permutation epimorphism, it in turn suffices to compute suitable maps .

###### Example 4.7.

Consider the situation in Example 4.5. Let and . Then the map is compatible with and induces the permutation epimorphism . The product maps and are isomorphisms of sets and permutation groups, respectively.

We illustrate how to construct one of the needed projections for PA type groups.

###### Example 4.8.

Let and . We denote by the trivial permutation group on . The subgroup is normal in . Let us denote sets of the form by . Then partitioning into orbits under yields the block system

Note how mapping each to the block of it is contained in is equivalent to mapping each to . Thus we have essentially constructed the map . Observe that we only used the group theoretic property that is a maximal normal subgroup of and thus in particular did not use the actual product structure of .

Analogously we can construct the map . For let be the permutation epimorphisms of induced by and , respectively. Since is an isomorphism, must be a monomorphism. By order arguments is thus an isomorphism.

In general the above construction does not yield permutation epimorphisms with
identical images.
We can alleviate this by computing elements of the given
group which conjugate the minimal normal subgroups of its socle to each other.
For the general construction see the
*(homogenized) product decomposition by minimal normal subgroups*
in [siccha:phd-thesis, Definitions 5.1.3 and 5.1.5].
Lemma 4.1 then follows from
[siccha:phd-thesis, Corollary 5.19].

## 5 Reduction Homomorphism

Recall from Section 2 that a key ingredient of our second phase is a group homomorphism which reduces the original problem on points to a problem on less or equal than points. We illustrate shortly how to construct this homomorphism, for the details refer to [siccha:phd-thesis, Theorem 9.1.6].

Let be a primitive group with non-regular socle
and be a
*socle-component of *, confer
[siccha:phd-thesis, Chapters 5 and 7] for a definition.
Then is a non-abelian
simple group,
there exists a positive integer
such that is isomorphic to ,
and
by
[siccha:phd-thesis, Lemma 2.6.1]
we have
for some .
Denote by the permutation group induced by the right-regular action of
on itself.
We show that we can evaluate the following two group
homomorphisms:
first an embedding
and second an epimorphism
,
where is the imprimitive wreath product
and thus acts on points.

We sketch the proof that . Let and . Note that for we have . Since is regular, we have . Since is a socle-component of , we have by [guralnick-maroti-pyber:normalizers-primitive-groups, Lemma 7.7]. In total we have .

In our implementation we use a modified version of this reduction. For groups of type PA we can directly compute an isomorphism from the product action wreath product into the corresponding imprimitive wreath product.

## 6 Implementation

A version of our normalizer algorithm for groups of type PA is implemented in the GAP package NormalizersOfPrimitiveGroups.

Table 1 shows a comparison of runtimes of our algorithm and the GAP function Normalizer. At the time of writing, there are two big bottlenecks in the implementation. First, the GAP built-in algorithm to compute the socle of a group is unnecessarily slow. State-of-the-art algorithms as in [cannon-holt:chief-series-composition-series-socle] are not yet implemented. Secondly, computing a permutation which transforms a given product decomposition into a so-called natural product decomposition currently also is slow. The latter may be alleviated by implementing the corresponding routines in for example C [c-lang] or Julia [julia-lang]. Note that the actual normalizer computation inside the normalizer of the socle appears to be no bottleneck: in the example with socle type it took only 40ms!

Socle type | Degree | Our algorithm | GAP built-in alg. |
---|---|---|---|

25 | 24ms | 200ms | |

125 | 50ms | 1500ms | |

625 | 300ms | 29400ms | |

78125 | 67248ms | – | |

36 | 40ms | 300ms | |

216 | 90ms | 1900ms | |

1296 | 400ms | 64000ms | |

49 | 38ms | 900ms | |

343 | 200ms | 16800ms | |

2401 | 1400ms | 839000ms |

## 7 Acknowledgments

Substantial parts of the work presented in this article were written while the author was supported by the German Research Foundation (DFG) research training group “Experimental and constructive algebra” (GRK 1632) and employed by the Lehr- und Forschungsgebiet Algebra, RWTH Aachen University and the Department of Mathematics, University of Siegen.

Comments

There are no comments yet.