 # A Polynomial-time Algorithm to Compute Generalized Hermite Normal Form of Matrices over Z[x]

In this paper, a polynomial-time algorithm is given to compute the generalized Hermite normal form for a matrix F over Z[x], or equivalently, the reduced Groebner basis of the Z[x]-module generated by the column vectors of F. The algorithm is also shown to be practically more efficient than existing algorithms. The algorithm is based on three key ingredients. First, an F4 style algorithm to compute the Groebner basis is adopted, where a novel prolongation is designed such that the coefficient matrices under consideration have polynomial sizes. Second, fast algorithms to compute Hermite normal forms of matrices over Z are used. Third, the complexity of the algorithm are guaranteed by a nice estimation for the degree and height bounds of the polynomials in the generalized Hermite normal form.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

The Hermite normal form (abbr. HNF) is a standard representation for matrices over principal ideal domains such as and , which has many applications in algebraic group theory, integer programming, lattices, linear Diophantine equations, system theory, and analysis of cryptosystems [5, 16, 20]. Efficient algorithms to compute HNF have been studied extensively until recently [5, 16, 9, 23, 22, 2, 14, 21]. Note that is not a PID and a matrix over cannot be reduced to an HNF. In , the concept of generalized Hermite normal form (abbr. GHNF) is introduced and it is shown that any matrix over can be reduced to a GHNF. Furthermore, a matrix is a GHNF if and only if the set of its column vectors forms a reduced Gröbner basis of the -module generated by in under certain monomial order. Similar to the concept of lattice , a -module in is called a -lattice which plays the same role as lattice does in the study of binomial ideals and toric varieties . For instance, the decision algorithms for some of the major properties of Laurent binomial difference ideals and toric difference varieties are based on the computation of GHNFs of the exponent matrices of the difference ideals . This motivates the study of efficient algorithms to compute the GHNFs.

The reduced Gröbner basis for a -lattice can be computed with the Gröbner basis methods for modules over rings [6, 15, 18]. However, such general algorithms do not take advatage of the special properties of -modules and do not have a complexity analysis. Also note that the worst case complexity of computing Gröbner bases in is double exponential .

The main contribution of this paper is to give an algorithm to compute the GHNF of a matrix or the reduced Gröbner basis of the -lattice generated by the column vectors of , which is both practically efficient and has polynomial bit size computational complexity. The algorithm consists of three main ingredients.

The first ingredient comes from the powerful idea in Faugère’s F4 algorithm  and the XL algorithm  of Courtois et al. To compute the Gröbner basis of the ideal generated by , these algorithms apply efficient elimination algorithms from linear algebra to the coefficient matrix of for certain . Although the F4 algorithm can not improve the worst case complexity, it is generally faster than the classical Buchberger algorithm . In this paper, to compute the GHNF of with columns , due to the special structure of the Gröbner bases in , we design a novel method to do certain prolongations such that the sizes of the coefficient matrices of those are nicely controlled.

The second ingredient is a nice estimation for the degree and height bounds of the polynomials in the GHNF of . We show that the degrees and the heights of the key elements of are bounded by and , respectively, where and are the maximal degree and maximal height of the polynomials in , respectively. Furthermore, we show that for a matrix and the degrees of the polynomials in are bounded by a polynomial in , which is a key factor in the complexity analysis of our algorithm. Note that the degree bound also depends on the coefficients of . The bounds about the GHNF are obtained based on the powerful methods introduced by Aschenbrenner in , where the first double exponential algorithm for the ideal membership problem in is given. In order to find the degree and height bounds for the GHNF, we need to find solutions of linear equations over , whose degree and height are bounded. Due to the special structure of the Gröbner basis in , we give better bounds than those in .

The third ingredient is to use efficient algorithms to compute the HNF for matrices over . The computationally dominant step of our algorithm is to compute the HNF of the coefficient matrices of those prolongations obtained in the first ingredient. The first polynomial-time algorithm to compute HNF was given by Kannan and Bachem  and there exist many efficient algorithms to compute HNFs for matrices over [9, 23, 22, 5] and matrices over [2, 14, 21]. Note that the GHNF for a matrix over cannot be recovered from its HNF over directly. In the complexity analysis of our algorithm, we use the HNF algorithm with the best bit size complexity bound .

The algorithm is implemented in Magma and Maple and their default HNF commands are used in our implementation. In the case of , our algorithm is shown to be more efficient than the Gröbner basis algorithm in Magma and Maple. In the general case, the proposed algorithm is also very efficient in that quite large problems can be solved.

The rest of this paper is organized as follows. In Section 2, we introduce several notations for Gröbner bases of lattices. In Section 3, we give degree and height bounds for the GHNF of a matrix over . In Section 4, we give the algorithm to compute the GHNF and analyze its complexity. Experimental results are shown in Section 5. Finally, conclusions are presented in Section 6.

## 2 Preliminaries

In this section, some basic notations and properties about Gröbner bases for lattices will be given. For more details, please refer to [6, 12, 15].

For brevity, a module in is called a lattice. Any lattice has a finite set of generators and this fact is denoted as . If , then we call a matrix representation of . If , is called a polynomial vector.

A monomial in is an element of the form , where , and is the canonical -th unit vector in . A term in is a multiplication of an integer and a monomial , that is . The admissible order on monomials in can be defined naturally: if or and . The order can be naturally extended to terms: if and only if or .

With the admissible order , any can be written in a unique way as a -linear combination of monomials,

,

where and . We define the leading coefficient, leading monomial, and leading term of as , , and , respectively.

The order can be extended to elements of in a natural way: for if and only if . We will use the order throughout this paper.

For two terms and in with , is called -reduced if one of the following conditions is valid: ; and ; or , , and . For any with , is called -reduced if any term of is -reduced. If is not -reduced, then by the reduction algorithm for the polynomials in  , one can compute a unique and a quotient such that is -reduced and is denoted as . If is -reduced, then set to be . For and with , is called -reduced if any term of is -reduced for . Let and for , set Denote and say is reduced to by .

###### Definition 2.1.

Let , . Then the S-vector of and is defined as follows: if then ; otherwise

 S(f,g)=⎧⎪ ⎪⎨⎪ ⎪⎩f−abxk−sg,if b|a;baf−xk−sg,if a|b;uf+vxk−sg,if a∤b and  b∤a, \emphwhere gcd(a,b)=ua+vb. (1)

If , the S-vector is called S-polynomial, which is the same with the definition in .

###### Definition 2.2.

A finite set is called a Gröbner basis for the lattice generated by if for any , there exists , such that . A Gröbner basis is called reduced if for any is -reduced. A Gröbner basis is called minimal if for any is -reduced.

It is easy to see that is a Gröbner basis if and only if for any . The Buchberger criterion for Gröbner basis is still true: is a Gröbner basis if and only if for all . Gröbner bases in this paper are assumed to be ranked in an increasing order with respect to the admissible order . That is, if is a Gröbner basis, then . To make the reduced Gröbner basis unique, we further assume that for any .

We need the following property for Gröbner bases in .

###### Proposition 2.3 ().

Let be the reduced Gröbner basis of a module in , , and . Then

1. .

2. and for .

3. for . Moreover, if is the primitive part of , then , for .

This proposition also applies to the minimal Gröbner bases. Here are three Gröbner bases in : , , .

For a polynomial set in , we denote by the GCD of the contents of and the primitive part of . Now, we give a refined description of Gröbner bases for ideals in .

###### Proposition 2.4 ().

with is a minimal Gröbner basis of in if and only if , , and , such that

1. ;

2. ;

3. is monic with degree , and ;

4. , and , for , where .

Next, we introduce the concept of generalized Hermite normal form. Let

 C=⎛⎜ ⎜ ⎜ ⎜ ⎜ ⎜⎝c11…c1,l1c1,l1+1………………………………………………cr1,1…cr1,l1cr1,l1+1…………………0…0cr1+1,1…cr1+1,l2…………………………………………0…0cr2,1…cr2,l2…………………………………………0…00…0…0crt−1+1,1…crt−1+1,lt……………………………0…00…0…0crt,1…crt,lt⎞⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠n×m (2)

whose elements are in . It is clear that and . Assume

,

and assume . Then the leading term of is , where is the -th column of .

###### Definition 2.5.

The matrix is called a generalized Hermite normal form (abbr.  GHNF ) if it satisfies the following conditions:

1)

for any .

2)

.

3)

can be reduced to zero by the column vectors of the matrix for any .

4)

is reduced with respect to the column vectors of the matrix other than , for any .

###### Theorem 2.6 ().

is a reduced Gröbner basis under the monomial order and if and only if the polynomial matrix is a  GHNF .

## 3 Degree and height bounds for the GHNF

We first give some notations. Let , where is a subring of . Denote by the maximal absolute value of the coefficients of . Let , with . For , let and .

For a prime , let be the local ring of at . For where is a unit in , let be the -adic valuation. Let be the completion[1, 10] of and the polynomial ring with coefficients in . Denote by the completion of [1, 10].

For any subring of or and in , let be the module generated by in .

### 3.1 Degree and height bounds in Z[x]

In this section, we give several basic degree and height bounds in . By the extended Euclidean algorithm, we have

###### Lemma 3.1.

Let be a field, , and . Then there exist with for any , satisfying .

In this section, we assume , and , unless specified otherwise explicitly.

###### Lemma 3.2.

If , then   for some with and some with degree . In this case, the height of the  GHNF of is .

###### Proof.

By Lemma 3.1, we have , where of degree . Assume . Then we have the matrix equation , where with

 Ai=⎛⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜⎝ai0ai1ai0⋮⋱ai,dai0⋱⋮ai,d⎞⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠2d×d

for , and . Let . By the Cramer’s rule, can be bounded by the nonzero minors of . By the Hadamard’s inequality, we have , where . So . In this case, . Hence, the height of  GHNF of is . ∎

The following lemma is given by Gel’fond  and a simpler proof can be found in [24, p178].

###### Lemma 3.3.

Let and be two monic polynomials in , such that . Then

The following lemma gives a height bound for the gcd in .

###### Lemma 3.4.

Let and in . Then the height of is bounded by .

###### Proof.

Since is in , for each , there exists a such that Let and . Then and . Let . By Lemma 3.3, we have for each , where . Then . We have

 height(g) ≤height(g)+height(gi) ≤12log(di+1)+dilog2+height(fi)  \emphforany$i$ (3) ≤12log(d+1)+dlog2+h.

###### Remark 3.5.

By equation (3.1), we have for any .

We now give the degree and height bounds for the  GHNF in .

###### Lemma 3.6.

Let and the GHNF of . Then and .

###### Proof.

Obviously, the degree bound of the  GHNF in is by the procedure of the Gröbner basis  computation. Let in , then, is the  GHNF of . By Lemmas 3.4 and 3.5, and are both . Moreover, . By Lemma 3.2, . So, . ∎

Finally, we consider an effective Nullstellensatz in , whose proof follows that of Lemma 6.4 in .

###### Lemma 3.7.

If , then there exist of degree at most such that .

###### Proof.

Suppose , then . By Lemma 3.2, there exist with height and with degrees satisfying

 δ=f1g1+⋯+fmgm. (4)

If is a unit in , then

 1=f1(g1/δ)+⋯+fm(gm/δ).

Let for . Then we have the required properties. Suppose that is not a unit. Let Clearly we have . Then by the Extended Euclidean Algorithm, there exist with

 1−(r1f1+⋯+rmfm)∈(p)Z(p)[x]

and for all So there exists and such that

 1−(f1s1+⋯+fmsm)=s. (5)

We have for all ; hence . By equations (4) and (5), we have

 1=f1s1+⋯+fmsm+s=f1h1+⋯+fmhm

with . We have

 deg(sgj)≤μ(2d−1)+d≤3μd.

Since , it follows that is bounded by . ∎

Then we can give the degree bound for the global case.

###### Lemma 3.8.

If , then there exist such that , with for .

###### Proof.

By Lemma 3.2, we have with degrees and satisfying

 δ=f1g1+⋯+fmgm.

Let be all the prime factors of . Since , we have . By Lemma 3.7, there exist with degrees and satisfying . Then there exist satisfying

 1=aδ+a1δ(p1)+⋯+akδ(pk).

Hence letting for , we get From this, we can easily get for . ∎

### 3.2 Degree and height bounds for solutions to linear equations over Z[x]

Throughout this section, let , the maximal degree of elements in , and the maximal height of elements in . For anysubring of , let

 SolR[x](F)={Y∈R[x]m|FY=0}

which is an -module in . Let be the rank of and the matrix consisting of linear independent rows of . Then, . So, we may assume unless mentioned otherwise. In this section, we will show that has a set of generators whose degrees and heights can be nicely bounded.

For a prime , is called regular of degree with respect to , or simply, regular of degree when there is no confusion, if its reduction is unit-monic of degree , that is,