# Homomorphism Extension

We define the Homomorphism Extension (HomExt) problem: given a group G, a subgroup M ≤ G and a homomorphism φ: M → H, decide whether or not there exists a homomorphism φ: G→ H extending φ, i.e., φ|_M = φ. This problem arose in the context of list-decoding homomorphism codes but is also of independent interest, both as a problem in computational group theory and as a new and natural problem in NP of unsettled complexity status. We consider the case H=S_m (the symmetric group of degree m), i.e., φ : G → H is a G-action on a set of m elements. We assume G< S_n is given as a permutation group by a list of generators. We characterize the equivalence classes of extensions in terms of a multidimensional oracle subset-sum problem. From this we infer that for bounded G the HomExt problem can be solved in polynomial time. Our main result concerns the case G=A_n (the alternating group of degree n) for variable n under the assumption that the index of M in G is bounded by poly(n). We solve this case in polynomial time for all m < 2^n-1/√(n). This is the case with direct relevance to homomorphism codes (Babai, Black, and Wuu, arXiv 2018); it is used as a component of one of the main algorithms in that paper.

## Authors

• 2 publications
• ### Two-closure of supersolvable permutation group in polynomial time

The 2-closure G of a permutation group G on Ω is defined to be the large...
12/21/2019 ∙ by Ilia Ponomarenko, et al. ∙ 0

• ### Extension of vertex cover and independent set in some classes of graphs and generalizations

We consider extension variants of the classical graph problems Vertex Co...
10/10/2018 ∙ by Katrin Casel, et al. ∙ 0

• ### Finding binomials in polynomial ideals

We describe an algorithm which finds binomials in a given ideal I⊂Q[x_1,...
07/07/2016 ∙ by Anders Jensen, et al. ∙ 0

• ### List-decoding homomorphism codes with arbitrary codomains

The codewords of the homomorphism code aHom(G,H) are the affine homomorp...
06/08/2018 ∙ by László Babai, et al. ∙ 0

• ### The Graph Isomorphism Problem: Local Certificates for Giant Action

This thesis provides an explanation of László Babai's quasi-polynomial a...
09/23/2019 ∙ by Tim Seppelt, et al. ∙ 0

• ### A subquadratic algorithm for the simultaneous conjugacy problem

The d-Simultaneous Conjugacy problem in the symmetric group S_n asks whe...
07/11/2020 ∙ by Andrej Brodnik, et al. ∙ 0

• ### On the AC^0[⊕] complexity of Andreev's Problem

Andreev's Problem states the following: Given an integer d and a subset ...
07/18/2019 ∙ by Aditya Potukuchi, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Homomorphism Extension asks whether a group homomorphism from a subgroup can be extended to a homomorphism from the entire group. We consider the case that the groups are represented as permutation groups. The complexity of this natural problem within NP is unresolved.

### 1.1 Connection to list-decoding homomorphism codes

Our study is partly motivated by our recent work on local list-decoding homomorphism codes from alternating groups [BBW18]. For groups and , the set of (affine) homomorphisms can be viewed as a code. The study of list-decoding such codes originates with the celebrated paper by Goldreich and Levin [GL89] and has more recently been championed by Madhu Sudan and his coauthors [GKS06, DGKS08, GS14]. While this body of work pertains to groups that are “close to abelian” (abelian, nilpotent, some classes of solvable groups), in [BBW18] we began the study of the case when the group is not solvable. As a test case, we have studied the alternating groups and plan to study other classes of simple groups.

For homomorphism codes, the “code distance” corresponds to the maximum agreement between two homomorphisms. The list-decoding efforts described in Babai, Black, and Wuu [BBW18] only guarantee returning partial homomorphisms, defined on subgroups of order . In the case of solvable groups (all previously studied cases fall in this category), maximum agreement sets are subgroups of smallest index111 Strictly speaking, this statement requires the “irrelevant kernel” to be trivial. The irrelevant kernel is the intersection of the kernels of all homomorphisms, cf. [BBW18, Section 4]. The solvablenilpotent case appears in [Guo15]., so is the only subgroup of of order greater than . This is not the case, however, for groups in general; in particular, it fails for the alternating groups where a maximum agreement set can be a subgroup of index (but not smaller). To solve the list-decoding problem, we need to extend these partial homomorphisms to full homomorphisms, i.e., we need to solve the Homomorphism Extension Search Problem for subgroups of order (and therefore, of small index). Indeed, a special case of the main result here (Theorem 1.9) is used, and is credited to this paper, in Babai, Black, and Wuu [BBW18] to complete the proof of one of the main results of that paper. For a more detailed explanation, see part (b) of Section 10, especially Remark 10.2.

### 1.2 Definition and results

We define the Homomorphism Extension problem. Denote by the set of homomorphisms from group to group .

###### Definition 1.1.

Homomorphism Extension
Instance: Groups and and a partial map .
Solution: A homomorphism that extends , i.e., .

The Homomorphism Extension Decision Problem (HomExt) asks whether a solution exists.

###### Remark 1.2.

Our algorithmic results for HomExt solve the Homomorphism Extension Search Problem as well, which asks whether a solution exists and, if so, to find one.

The problems as stated above are not fully specified. Representation choices of the groups and affect the complexity of the problem. For example, may be given as a permutation group, a black-box group, or a group given by a generator-relator presentation.

For the rest of this paper we restrict the problem to permutation groups.

###### Definition 1.3.

HomExtPerm is the version of HomExt where the groups are permutation groups given by a list of generators. HomExtSym is the subcase of HomExtPerm where the codomain is a symmetric group.

Membership in permutation groups is polynomial-time testable. Our standard reference for permutation group algorithms is [Ser03]. Section 8 summarizes the results we need, including material not easily found in the literature. Our standard reference for permutation group theory is [DM96].

Partial maps are represented by listing their domain and values on the domain. Homomorphisms in are represented by their values on a set of generators of .

For a partial map , we denote by the subgroup of generated by the domaim of .

###### Remark 1.4.

Whether the input map extends as a homomorphism in is a polynomial-time testable condition in permutation groups. See Section 8.3.

Since extending to is easy, this paper is primarily concerned with extending a homomorphism from a subgroup to a homomorphism from the whole group.

###### Assumption 1.5 (Given partial map defines a homomorphism on subgroup).

Unless otherwise stated, in our analysis we assume without loss of generality that the input partial map extends to a homomorphism in . This is possible due to Remark 1.4. In this case, the homomorphism is represented by , as a partial map on generators of . We will think of as the input to HomExt. We often drop the subscript on .

Since a minimal set of generators of a permutation group of degree has no more than elements [Bab86] and any set of generators can be reduced to a minimal set in polynomial time, we shall assume our permutation groups are always given by at most generators.

We note that the decision problem HomExtPerm is in NP.

###### Open Problem 1.6.

Is HomExtPerm NP-complete?

This paper considers the important subcase of the problem when , the symmetric group of degree . A homomorphism is called a group action (more specifically, a -action) on the set .

The HomExtSym problem seems nontrivial even for bounded (and variable ).

###### Theorem 1.7 (Bounded domain).

If has bounded order, then HomExtSym can be solved in polynomial time.

The degree of the polynomial in the polynomial running time is exponential in .

###### Open Problem 1.8.

Can HomExtSym be replaced by HomExtPerm in Theorem 1.7, i.e., can be replaced by ?

Our main result, the one used in our work on homomorphism codes, concerns variable and is stated next.

In the results below, “polynomial time” refers to time.

###### Theorem 1.9 (Main).

If (alternating group of degree ), HomExtSym can be solved in polynomial time under the following assumptions.

• The index of in is bounded by , and

• , where .

Under the assumptions above, counting the number of extensions is also polynomial-time.

###### Theorem 1.10 (Main, counting).

Under the assumption of Theorem 1.9, the number of solutions to the instance of HomExtSym can be found in polynomial time.

Note the rather generous upper bound on in item (ii). Whether an instance of HomExtSym satisfies the conditions of Theorem 1.9 can be verified in time (see Section 8.3).

We state a polynomial-time result for very large (Theorem 1.11, of which Theorem 1.7 is a special case).

###### Theorem 1.11 (Large range).

If and , then HomExtSym can be solved in polynomial time.

### 1.3 Methods

We prove the results stated above by reducing HomExtSym to a polynomial-time solvable case of a multi-dimensional oracle version of Subset Sum with Repetition (SSR). SSR asks to represent a target number as a non-negative integral linear combination of given numbers, whereas the classical Subset Sum problem asks for a 0-1 combination. SSR is NP-complete by easy reduction from Subset Sum.

We call the multi-dimensional version of the SSR problem MultiSSR. The reduction from homomorphism extension to MultiSSR is the main technical contribution of the paper (Theorem 1.12 below).

The reduction is polynomial time; therefore, the complexity of our solutions to HomExtSym will be the complexity of special cases of MultiSSR that arise. The principal case of MultiSSR is one we call “triangular” ; this case can be solved in polynomial time. The difficulty is aggravated by exponentially large input to MultiSSR, to which we assume oracle access (OrMultiSSR Problem). Implementing oracles calls will amount to solving certain problems in computational group theory, addressed in Section 8 of the Appendix.

The MultiSSR problem takes as input a multiset in universe (viewed as a non-negative integral function ) and a set of multisets in . MultiSSR asks if is a nonnegative integral linear combination of multisets in (see Section 4.2). The set will be too large to be explicitly given (it will contain one member per conjugacy class of subgroups of ). Instead, we contend with oracle access to the set . For a more formal presentation of MultiSSR and OrMultiSSR, see Section 3.

From every instance of HomExtSym describing a group action, we will construct an OrMultiSSR instance (see Section 4.2). In the next result, we describe the merits of this translation.

Two permutation actions are permutation equivalent if there exists such that for all .

###### Theorem 1.12 (Translation).

For every instance , the instance of OrMultiSSR satisfies the following.

1. can be efficiently computed from . For what this means, see Section 4.2.

2. There exists a bijection between the set of non-empty classes of equivalent (under permutation equivalence) extensions and the set of solutions to .

3. Given a solution to , a representative of the equivalence class of extensions can be computed efficiently.

Here, “efficiently” means in -time. The universe of will be the conjugacy classes of subgroups of . The set will be indexed by the conjugacy classes of subgroups of . These sets can be exponentially large. For , by [Pyb93].

Now, it suffices to efficiently find solutions to instances of OrMultiSSR arising under this reduction.

Theorem 1.11 (large ) follows from Theorem 1.12 and a result of Lenstra [Len83] (cf. Kannan [Kan87]), that shows Integer Linear Programming is fixed-parameter tractable. As MultiSSR can naturally be formulated as an

integer linear program, we conclude polynomial-time solvability due to the assumed magnitude of

(see Appendix, Section 7).

For Theorem 1.9, we will show that instances satisfy the conditions of TriOrMultiSSR, a “triangular” version of OrMultiSSR (see Section 5).

###### Theorem 1.13 (Reduction to TriOrMultiSSR).

If an instance of HomExtSym satisfies the conditions of Theorem 1.9, the instance of OrMultiSSR is also an instance of TriOrMultiSSR. The oracle queries can be answered in polynomial time.

Despite only being given oracle access, TriOrMultiSSR turns out to be polynomial-time solvable (see Section 3.2, or the Appendix, Section 5).

###### Proposition 1.14.

TriOrMultiSSR can be solved in polynomial time.

###### Proposition 1.15.

If a solution to TriOrMultiSSR exists, then it is unique.

Polynomial time for an OrMultiSSR problem means polynomial in the length of and the length of the representation of elements of . For details on representating multisets, see Section 2.1.

### 1.4 Efficient enumeration

The methods discussed give a more general result than claimed. Instead of solving the Search Problem, we can in fact efficiently solve the Threshold- Enumeration Problem for HomExtSym. This problem asks to find the set of extensions, unless there are more than , in which case output of them.

This question is also motivated by the list-decoding problem; specifically, Threshold-2 Enumeration can be used to prune the output list. See Section 10 for details. We remark that solving Threshold-2 Enumeration already requires all relevant ideas in solving Threshold- Enumeration.

###### Definition 1.16 (Threshold-k).

For a set and an integer , the Threshold- Enumeration Problem asks to return the following pair of outputs.
If , return and
Else, return “more” and a list of distinct elements of .

Note that the Threshold- Enumeration Problem is simply the decision problem “is non-empty?” while the Threshold- Enumeration Problem includes the search problem (if not empty, find an element of ).

We say that an algorithm efficiently solves the Threshold- Enumeration Problem if the cost divided by is considered “modest” (in our case, polynomial in the input length).

Our work on list-decoding homomorphism codes uses solutions to the Threshold- Enumeration Problem for the set of extensions of a given homomorphism. With potential future applications in mind, we discuss the Threshold- Enumeration Problem for variable .

###### Definition 1.17.

Homomorphism Extension Threshold- Enumeration (HomExtThreshold) is the Threshold- Enumeration Problem for the set of solutions to Homomorphism Extension ( defined below).

###### Notation 1.18 (HExtG(ψ)).

We will denote by the set of solutions to an instance of HomExt.

 HExtG(ψ):={φ∈Hom(G,H):φ|M=ψ}.

The following condition strengthens the notion of efficient solutions to threshold enumeration.

###### Definition 1.19 (Efficient enumeration).

We say that a set can be efficiently enumerated if an algorithm lists the elements of at modest marginal cost.

The marginal cost of the -th element is the time spent between producing the -st and the -th elements. In this paper, “modest marginal cost” will mean marginal cost, where and denote the degrees of the permutation groups and , respectively.

###### Observation 1.20.

If a set can be efficiently enumerated then the Threshold Enumeration Problem can be solved efficiently.

In particular, the decision and search problems can be solved efficiently. The following theorems are the strengthened versions of the ones stated in Section 1.2.

###### Theorem 1.21 (Bounded domain, enumeration).

If has bounded order, then the set can be efficiently enumerated.

###### Theorem 1.22 (Main, enumeration).

If (alternating group of degree ), then the set can be efficiently enumerated under the following assumptions:

• the index of in is bounded by poly, and

• , where .

###### Theorem 1.23 (Large range, enumeration).

If and , then the HomExtSym Threshold- Enumeration Problem can be solved in ) time.

### 1.5 Enumeration methods

Recall that Theorem 1.12 gave a bijection between classes of equivalent extensions and solutions to the OrMultiSSR instance. It remains to solve the Threshold- Enumeration Problem for OrMultiSSR, then to efficiently enumerate extensions within one equivalence class, given a representative of that class.

#### Solutions of Threshold-k for OrMultiSSR

Under the assumptions of Theorem 1.9, the instance of OrMultiSSR (reduced to from the HomExt instance ) will also be an instance of TriOrMultiSSR. Since solutions are unique if they exist (Proposition 1.15), solving the Search Problem also solves the Threshold- Enumeration Problem for TriOrMultiSSR. But, the Search Problem can be solved in polynomial time by Proposition 1.14.

In the case of Theorem 1.7, is an integer linear program with a bounded number of variables and constraints (corresponding to classes of subgroups of ) and the solutions can therefore be efficiently enumerated.

For Theorem 1.23 (thus also implying Theorem 1.21), the Threshold- Enumeration Problem for the Integer Linear Program version of can be answered in polynomial time by viewing it as an integer linear program. See Section 7.

#### Efficient enumeration within one equivalence class

We now wish to efficiently enumerate extensions within each class of equivalent extensions, given a representative.

Two permutation actions are equivalent (permutation) actions if there exists such that for all . We say that two homomorphisms are equivalent extensions of the homomorphism if they (1) both extend and (2) are equivalent permutation actions.

Enumerating extensions within one equivalence class reduces to the following: Given subgroups , efficiently enumerate coset representatives for in .

This problem was solved by Blaha and Luks in the 1980s (unpublished, cf. [BL94]). For completeness we include the solution based on communication by Gene Luks [Luk] (see Section 9).

We explain the connection between finding coset representatives and the classes of equivalent extensions of . Consider an extension of . For any , the homomorphism , defined as for all , is an equivalent permutation action. First, if and only if (the centralizer in of the -image of , i.e., the set of elemenets of that commute with all elements in ). The centralizer of a group in the symmetric group can be found in polynomial time (see Section 8.4). Also, extends (thus is an equivalent extension to ) if and only if .

So, finding coset representatives of in suffices for finding all equivalent extensions. Applying the Blaha–Luks result yields the following corollary (see Section 6).

###### Corollary 1.24.

Let and . Suppose that extends . Then, the class of extensions equivalent to can be efficiently enumerated.

### 1.6 Acknowledgments

I would like to thank Madhu Sudan for introducing me to the subject of list-decoding homomorphism codes. I would also like to thank Gene Luks for communicating the content of Section 9. Last but not least, I would like to thank my adviser Laci Babai for his generous support, ideas, and endless advice.

## 2 Preliminaries

We write for .

### 2.1 Multiset notation

In this paper, we will consider both sets and multisets. All sets and multisets are finite.

We typographically distinguish multisets using “mathsf” font, e.g., , and denote multisets. A multiset within a universe is formally a function . For a member of the universe, the multiplicity of in is . We say that is an element of () if , i.e., if has non-zero multiplicity in . The set of elements of is called the support of , denoted by . We algorithmically represent a multiset by listing its support and the values on the support, so the description is of length , where is the description length for elements of . The size of is , the 1-norm of the function .

Let be two multisets in the same universe. Their sum is the multiset obtained by adding the multiplicities. We say that is a submultiset of if for all .

Sets will continue to be denoted by standard font and defined via one set of braces . Often it is convenient to list the elements of a multiset as using double braces, where and each occurs times in this list. The length of this list is the size of . In our notation, but .

A disjoint union of two sets is denoted by .

### 2.2 Group theory notation

Let be a group. We write to express that is a subgroup; we write to denote that is a normal subgroup.

For and , we call the coset of a subcoset of . We define the index of a subcoset in by . For a subset of a group , we denote by the subgroup generated by .

We introduce nonstandard notation for that will be used in the rest of the paper.

###### Notation 2.1 (Sub(G)).

We denote the set of subgroups of by .

For , denote by the (right) coset space (set of right cosets). For , denote by the set of double cosets. Double cosets form an uneven partition of . They are important in defining the MultiSSR instance from an instance of HomExtSym (see Section 4).

Two subgroups are conjugate in if there exists such that . The equivalence relation of conjugacy in is denoted by , or if is understood.

###### Notation 2.2.

For a subgroup , the conjugacy class of in is denoted by (or if is understood), so .

###### Notation 2.3 (Conj(G)).

We denote the set of conjugacy classes of by .

Using the introduced notation, if , then , and .

### 2.3 Permutation groups

In this section we fix terminology for groups and, in particular, permutation groups. A useful structure theorem for large subgroups of the alternating groups is presented as well. For reference see [DM96].

For a set , denotes the symmetric group on and denotes the alternating group on . Often, we write (or ) for the symmetric (or alternating) group on .

###### Definition 2.4 (Group actions).

A (permutation) action of a group on a set is given by a homomorphism , often denoted by or .

Let , , , and .

The image of under is denoted by . This notation extends to sets. So, and . The subset is -invariant if . The orbit of under action by is given by . The orbits of are -invariant and they partition . All -invariant sets are formed by unions of orbits.

The point stabilizer of is the subgroup of fixing , given by . The pointwise stabilizer of is the subgroup fixing every point in , given by . The setwise stabilizer of is given by .

Let be -invariant. For , denote by the restriction of the action of to . The group is the image of the permutation representation of in its action on . We see that .

We state a result that goes back to Jordan. Its modern formulation by Liebeck (see [DM96, Theorem 5.2A]) describes the small index subgroups of . This theorem is used to categorize group actions by in Theorem 1.9.

###### Theorem 2.5 (Jordan–Liebeck).

Let and let be an integer with . Suppose that has index . Then, for some with , we have .

### 2.4 Equivalent extensions

In this section we characterize equivalence of two group actions and, in particular, fix notation to describe equivalence.

###### Definition 2.6 (Equivalent permutation actions).

Two permutation actions and are equivalent if there exists a bijection such that for all and .

Note that two permutation actions of on the same domain are equivalent if there exists such that for all .

The Introduction defined two homomorphisms as “equivalent extensions” of if they both extend and if they are equivalent as actions. The following definition is equivalent to that definition provided in the Introduction.

For groups , the centralizer of in is given by .

###### Definition 2.7 (Equivalent extensions).

Let and . We say that and are equivalent extensions of if there exists such that for all .

Next we consider the equivalence of transitive group actions, through their point stabilizers. A -action on is transitive if for all , i.e., for every pair , there is a group element satisfying . Lemma 2.8 is Lemma 1.6A in [DM96].

###### Lemma 2.8.

Suppose acts transitively on the sets and . Let be the stabilizer of a point in the first action. Then, the actions are equivalent if and only if is the stabilizer of some point in the second action.

Recall that we denote the conjugacy class of a subgroup by , so is conjugate to if and only if . We find all point stabilizers are conjugate, and all conjugate subgroups are point stabilizers.

###### Fact 2.9.

Let be a point stabilizer of a transitive -action on . A subgroup is conjugate to () if and only if is also the stabilizer of a point in .

All transitive -actions are equivalent to one of its natural actions on cosets, defined below.

###### Example 2.10 (Natural actions on cosets).

For , we denote by the natural action of on . More specifically, an element acts on a coset as .

We see that the equivalence class of a transitive action is determined by the conjugacy class of its point stabilizers.

###### Corollary 2.11.

Consider a transitive -action . Let . The following are equivalent.

1. is equivalent to .

2. is a point stabilizer of the -action.

3. Some satisfying is a point stabilizer of the -action.

4. is equivalent to for .

Motivated by Corollary 2.11, we will define the notion of “-actions,” which describe transitive -actions up to equivalence. This definition will be generalized to intransitive actions (see Section 4.1).

### 2.5 Computation in permutation groups

A permutation group is given by a list of generators. We say that is known if a list of generators of is known. Based on this representation, membership testing can be performed in polynomial time. In Appendices 8 and 9 we list the algorithmic facts about permutation groups used in this paper.

## 3 Multi-dimensional subset sum with repetition

We consider the Subset Sum Problem with Repetitions (SSR). An instance is given by a set of positive integers and a “target” positive integer . The question is “can be represented as a non-negative linear combination222Notice that a non-negative linear combination of a set of integers is exactly the sum of a multiset in that set of integers. This question is asking for the existence of a multiset. of the other integers?” This problem is NP-complete by an easy reduction from the standard Subset Sum problem, which asks instead for a 0-1 linear combination.

We define a multi-dimensional version (MultiSSR) below. It has its own associated Decision, Search, and Threshold- Enumeration (Definition 1.16) Problems.

###### Definition 3.1.

Multi-dimensional Subset Sum with Repetition (MultiSSR)
Instance: Multiset and set of multisets in .333 is the underlying universe. Its entirety is not required in the input, but its size is the dimensionality of this problem. An element is a multiset in .
Solution: A multiset of summing to , i.e., a multiset satisfying .

###### Notation 3.2 (SubSum(K,F)).

We write for the set of solutions to an instance of MultiSSR, i.e.,

 SubSum(K,F):={L:F→N∣∣∣∑F∈FL(F)⋅F=K}.

The MultiSSR Decision Problem asks whether a solution exists ( is nonempty).

The MultiSSR Search Problem asks whether a solution exists and, if so, find one.

The MultiSSR Threshold- Enumeration Problem asks for the solution to the Threshold- Enumeration Problem for the set .

###### Remark 3.3 (MultiSSR as Integer Program).

Every instance of MultiSSR can naturally be viewed as an instance of Integer Linear Programming, with constraints and variables. The variables are the number of copies of each in the subset sum. The constraints correspond to checking that every element in has the same multiplicities in and .

### 3.1 Oracle MultiSSR

In our application, the set and universe will be prohibitively large to input explicitly. To address this, we define an oracle version of MultiSSR called Oracle Multi-dimensional Subset Sum with Repetitions (OrMultiSSR). We will reduce a HomExtSym instance to an OrMultiSSR instance denoted by , then show that the oracles can be answered efficiently.

We will find it convenient to introduce a bijection between and another set of simpler objects, used to index .444The index set will be the conjugacy classes of subgroups of , whereas will be a set of multisets of conjugacy classes of subgroups of . Access to is given by the oracle “,” which on input returns the element of indexed by . Elements of the universes and are encoded by strings in and , respectively, and the alphabets and encoding lengths constitute the input.

We allow non-unique555In our application, and . The universes and will be conjugacy classes of large subgroups of and , respectively. Each conjugacy class is non-uniquely encoded by generators of a subgroup in the class. encodings of and , but provide “equality” oracles.666We will not need to test membership of a string from in the universe. To handle non-unique encodings of in , we assume that returns the same multiset on (though possibly via different encodings) when handed different encodings of the same . Writing implies that is represented as a multiset on but with the promise that all strings in its support are encodings of elements of .

###### Definition 3.4.

Oracle Multi-dimensional Subset Sum with Repetition (OrMultiSSR)
Instance:
Explicit input
Alphabets and ;
Numbers , in unary; and
Multiset , by listing the elements in its support and their multiplicities.
Oracles
oracle for equality in or , and
oracle for the set , indexed by .
Solution: A sub-multiset of that defines a sub-multiset of summing to , i.e.,
a multiset satisfying .

###### Notation 3.5 (SubSum(K,F)).

Again, we write for the set of solutions to an instance of OrMultiSSR, though the indexing is slightly different.

 SubSum(K,F):={L:U→N∣∣∣∑v∈VL(v)⋅Fv=K}.

The length of the input is .

Due to non-unique encodings, checking whether a multiset satisfies will actually require calling the oracle, as the multisets on the left and right sides of the equation may be encoded differently.

### 3.2 Triangular MultiSSR

The Search Problem for OrMultiSSR with an additional “Triangular Condition” (and oracles corresponding to this condition) can be solved in polynomial time. We call this problem TriOrMultiSSR. This section defines TriOrMultiSSR. The next section will provide an algorithm that solves the TriOrMultiSSR Search Problem in polynomial time, proving Proposition 1.14.

Under the conditions of Theorem 1.9 (, has polynomial index, and the codomain has exponentially bounded permutation domain size ), a HomExtSym instance reduces to an instance of OrMultiSSR that satisfies the additional assumptions of TriOrMultiSSR. The additional oracles of TriOrMultiSSR can be efficiently answered (see Section 5).

#### Definition of TriOrMultiSSR

The triangular condition roughly says that the matrix for the corresponding (prohibitively large) integer linear program is upper triangular.

Below we say that a relation is a total preorder if it is reflexive and transitive with no incomparable elements.777A total order also imposes antisymmetry, i.e., if and then . That is the assumption we omit.

###### Definition 3.6.

Triangular Oracle Multi-dimensional Subset Sum with Repetition (TriOrMultiSSR)
Input, Set, Oracles, Output: Same as OrMultiSSR.
Triangular Condition: has a total preorder .
For every , the multiset contains a unique -minimal element .
The map is injective.
: compares two elements of , and
inverts , i.e., on input it returns

 △(u)={the unique v∈V such that τ(v)=u if v existsError if no such v exists. (1)

#### Integer program and uniqueness of solutions

Uniqueness of solutions for TriOrMultiSSR can be seen by looking at the integer linear program formulation, where variables correspond to and constraints correspond to . The Triangular Condition implies that, for every variable (), there exists a unique minimal constraint () containing this variable. The ordering on gives an ordering on by setting when . Order the variables and constraints by and , respectively (break ties in arbitrarily and have respect the tie-breaking of ). The matrix for the corresponding linear program is upper triangular.

Hence, if the integer program has a solution, it is unique. It trivially follows that solving the TriOrMultiSSR Search Problem also solves the corresponding Threshold- Enumeration Problem.

### 3.3 TriOrMultiSSR Search Problem

Algorithm 1 (TriOrMultiSSR) below solves the TriOrMultiSSR Search Problem in polynomial time (Proposition 1.14). If viewing the problem as a linear program, the algorithm essentially solves the upper triangular system of equations by row reduction, except that the dimensions are too big and only oracle access is provided.

In each iteration, TriOrMultiSSR finds one minimal element in . It removes the correct number of copies of from , in order to remove all copies of from . If this operation fails, the algorithm returns ‘no solution.’ Meanwhile, is updated in each iteration to record the number of copies of removed.

There are three reasons the operation may fail. (1) Removing all copies of from may not be possible through removal of (the number of copies is not an integer). (2) may not contain copies of (the operation results in negative values). (3) returns ( is not in the range of ).

#### Subroutines

: takes as input a subset and outputs one minimal element under . Using the oracle, a call can be executed in -time.

: Remove takes as input multisets and a nonnegative integer . It returns after removing copies of the multiset if possible, while accounting for non-unique encodings. Otherwise, it returns ‘no solution.’ Pseudocode for Remove is provided below.

: Consolidate adjusts for non-unique encodings of multisets as multisets. Given input the encoded multisets , Consolidate outputs multisets that encode the same multisets of , but uniquely. In other words, satisfy , with their combined support containing at most one encoding per element of .

#### Algorithm

Recall that we denote the empty multiset by . We give pseudocode for the Remove subroutine, followed by the main algorithm.

procedure Remove()
Remove duplicate encodings within .
Execute as , assuming integer range
if  has negative values then
return ‘no solution’
elsereturn
end if
end procedure

#### Analysis

The pre-processing step of Line 3 can be computed in time , by pairwise comparisons. The while loop of Line 4 is executed exactly number of times, for each .

The Consolidate call in TriOrMultiSSR returns , a different encoding of the multiset of , such that all elements of are uniquely encoded. This requires pairwise comparisons, or, calls to the oracle. Similarly, the Consolidate call in Remove can be achieved in calls to the oracle.

## 4 Reduction of HomExtSym to OrMultiSSR

We define the reduction from HomExtSym to OrMultiSSR then prove the three parts of Theorem 1.12: the polynomial-time efficiency of the reduction, the bijection between classes of equivalent extensions in and the set of solutions to , and efficiency of of defining an extension homomorphism from a solution .

For notational convenience, Section 4.1 defines “-actions” which describe permutation actions up to equivalence.

Towards proving Theorem 1.12 (a), Section 4.2 presents the reduction from a HomExtSym instance to the OrMultiSSR instance . We define the instance and show that its oracles can be answered in -time.

Section 4.3 proves the bijection claimed in Theorem 1.12 (b), assuming the transitive case. The transitive case is proved in Sections 4.4 and 4.5.

Section 4.6 proves Theorem 1.12 (c) by providing the algorithmic details of defining given a solution in .

### 4.1 (G,L)-actions, equivalence classes of G-actions

We introduce the terminology “-actions” (or “-actions” for transitive actions), which describes group actions up to permutation equivalence. The denotes a multiset of subgroups of , describing point stabilizers of the action. We make this more precise.

Recall that we write to denote the conjugacy class of the subgroup in .

###### Definition 4.1 ((G,l)-action).

Let be a transitive action. Let . We say that is a -action if is equivalent to , the natural on right cosets of (Example 2.10). We say that is a -action if is a -action.

By Corollary 2.11, a -action is a -action if and only if is a point stabilizer of the action. Moreoever, a -action is a -action if and only if . So, we can speak of -actions and make no distinction between -actions and -actions.

We now introduce notation to describe equivalence between intransitive actions.

###### Definition 4.2 ((G,L)-action).

Let be a group action. Let be a multiset listed as . We say the action of on is a -action if the orbits in of the action can be labeled so that acts on as a -action for all .888The multiset contains one point stabilizer per orbit of the -action. Viewing as a multiset is essential. For example, describes the trivial action of on one point, whereas describes the trivial action of on two points.

Again, the equivalence class of the -action is determined by the multiset up to conjugation of its elements. We introduce notation describing conjugate multisets.

###### Notation 4.3.

Let be a multiset of subgroups of . We denote by the multiset of conjugacy classes for the subgroups of .

In other words, for a multiset , denote by the multiset found by replacing every element by . Multiplicities of subgroup conjugacy classes in the multiset satisfy . We may write for if is understood.

###### Definition 4.4 (Conjugate multisets).

We say that two multisets are conjugate if . In other words, there exists a bijection such that for all .999This definition does not require conjugacy of all pairs simultaneously via the one element of .

Conjugate multisets describes group actions up to equivalence, as we see in the following next statement, which follows from the definitions and Corollary 2.11.

###### Corollary 4.5.

Let . The following are equivalent.

• and are conjugate, or .

• A -action is permutation equivalent to a -action.

• A -action is also a -action.

So, we can speak of -actions and make no distinction between -action and -actions.

### 4.2 Reduction

In this section, we discuss the -time reduction from HomExtPerm to OrMultiSSR.

###### Remark 4.6 (Meaning of “reduction”).

As usual, our reduction will compute the explicit inputs to OrMultiSSR from a HomExtSym instance in time. However, to account for the oracles in OrMultiSSR, we provide also answers to its oracles in -time.

Recall that denotes the set of subgroups of and denotes the set of conjugacy classes of subgroups of . Denote by the set of subgroups of with index bounded by . Denote by the set of conjugacy classes of subgroups of with index bounded by .

#### Construction of OMSψ

We define and encodings of the OrMultiSSR instance .

: .

: .

Encoding of : words of length over alphabet . A conjugacy class in of subgroups is encoded by a representative subgroup in , which is then encoded by a list of at most generators.

Encoding of : Likewise, with and .

: Let be a multiset containing one point stabilizer per orbit of the action . So, is a multiset of conjugacy classes, as in Notation 4.3.

#### Notational issues.

Using versus reflects the non-unique encoding of by and by , adhering to Notation 2.2 and 4.3. A conjugacy class will be encoded by . A multiset will be encoded by .

#### Calculating [K]

Calculating from : Decompose into its -orbits under the action described by . Choose one element per orbit.101010The choice of will not affect the correctness of the reduction. Then, calculate the multiset by finding the point stabilizer of each chosen element. So, calculating can be accomplished in -time by Proposition 8.3.

The oracle: given two subgroups in , check their conjugacy. This can be accomplished in -time by Proposition 8.5.

The set is indexed by . takes as input (represented by a ) and returns (represented by ), defined below. The multiset is defined so that -actions induce -actions.

###### Definition 4.7 (FL(σ)).

Let be a list of double coset representatives for . We define the multiset by

 FML(σ)=FL:={{σ−1iLσi∩M:i=1…d}}.

In the context of extending an -action to a -action, is understood, so we drop the superscript and write .

is well-defined. First of all, the choice of double coset representatives will not affect the conjugacy class of (see Remark 4.17). Moreover, if then . Section 4.4 further discusses and proves these claims about the properties of .

can be answered in -time by Proposition 8.6.

### 4.3 Combinatorial condition for extensions

We are now equipped to state the central technical result. It relates -actions to extension -actions by describing how -orbits may be grouped to form -orbits.

First, we address the case of transitive extensions.

As in Definition 4.7, denotes the multiset returned by the oracle on input . Since we assume the extension -action is transitive, the multiset describes exactly the -orbits that must be collected to form one -orbit.

###### Lemma 4.8 (Characterization of transitive extensions).

Let and . Let be an -action. Under these circumstances, extends to a -action if and only if is a -action.

The forward and backwards directions are Corollary 4.16 and Proposition 4.21 in the next two sections.

###### Remark 4.9.

To rephrase Lemma 4.8, an -action extends to a transitive -action if and only if (see Corollary 4.5).

The following result on intransitive actions is a corollary to Lemma 4.8.

Let