# On the security of a Loidreau's rank metric code based encryption scheme

We present a polynomial time attack of a rank metric code based encryption scheme due to Loidreau for some parameters.

## Authors

• 1 publication
• 15 publications
07/14/2020

### Extending Coggia-Couvreur Attack on Loidreau's Rank-metric Cryptosystem

A recent paper by Coggia and Couvreur presents a polynomial time key-rec...
12/23/2021

### An analysis of Coggia-Couvreur attack on Loidreau's rank-metric public key encryption scheme in the general case

In this paper we show that in the case where the public-key can be disti...
11/29/2019

### RAMESSES, a Rank Metric Encryption Scheme with Short Keys

We present a rank metric code-based encryption scheme with key and ciphe...
11/16/2021

### An analogue of the ElGamal scheme based on the Markovski algorithm

We give an analogue of the ElGamal encryption system based on the Markov...
04/07/2018

### An attack on a NIST proposal: RankSign, a code-based signature in rank metric

RankSign is a code-based signature scheme proposed to the NIST competiti...
12/14/2021

### A code-based hybrid signcryption scheme

A key encapsulation mechanism (KEM) that takes as input an arbitrary str...
11/02/2020

### Computing Power, Key Length and Cryptanalysis. An Unending Battle?

There are several methods to measure computing power. On the other hand,...
##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## Introduction

To instantiate McEliece encryption scheme, one needs a family of codes with random looking generator matrices and an efficient decoding algorithm. If the original proposal due to McEliece himself [12] relies on classical Goppa codes endowed with the Hamming metric, one can actually consider codes endowed with any other metric. The use of –linear rank metric codes, first suggested by Gabidulin et. al. [7] is of particular interest, since the –linearity permits a very “compact” representation of the code and hence permits to design a public key encryption scheme with rather short keys compared to the original McEliece proposal.

Compared to the Hamming metric world, only few families of codes with efficient decoding algorithms are known in rank metric. Basically, the McEliece scheme has been instantiated with two general families of rank metric codes, namely Gabidulin codes [5, 6] and LRPC codes [8].

In [11]

, Loidreau proposed the use of codes which can somehow be regarded as an intermediary version between Gabidulin codes and LRPC codes. These codes are obtained by right multiplying a Gabidulin code with an invertible matrix whose entries are in

and span an –subspace of small dimension . This approach can be regarded as a “rank metric” counterpart of BBCRS scheme [1] in Hamming metric.

In the present article, we explain why the case and is weak and describe a key recovery attack in this situation.

#### Note.

The material of the present article has been communicated to Pierre Loidreau in April 2016. The article [11] is subsequent to this discussion and proposes parameters which avoid the attack described in the present article.

## 1 Prerequisites

### 1.1 Rank metric codes

In this article denote positive integers and a prime power. A code of dimension is an –subspace of whose dimension as an

vector space is

. Given a vector , the rank weight or rank of , denoted as is the dimension of the –vector sub-space of spanned by the entries of . The support of a vector , denoted is the –vector space spanned by the entries of . Hence the rank of is nothing but the dimension of its support. The rank distance or distance of two vectors is defined as

 dR(x,y)def=|x−y|R.

Given a code , the minimum distance of is defined as

 dmin(C)def=minx∈C∖{0}{|x|R}.
###### Remark 1

Note that rank metric codes can be defined in a more general setting as subspace of a space of matrices or a space of morphisms between two vector spaces. A code can be regarded as a space of matrices by choosing an –basis of and associating to each vector the matrix whose –th column is the decomposition of the entry in the basis. A major difference between general spaces of matrices and the codes we introduced is that our vector spaces have an –linear structure. We chose to limit our presentation to codes of the form since only these codes are the object of the study in the present article.

### 1.2 q–polynomials and Gabidulin codes

A –polynomial or a linear polynomial is an –linear combination of monomials Such a polynomial induces a function which is –linear. The –degree of a –polynomial , denoted by is the integer such that the degree of is . In short:

 P=degqP∑i=0piXqi, pi∈Fqm, pdegqP≠0.

The following very classical result is crucial in what follows.

###### Proposition 1

Let be a –polynomial. Then, the set of roots of in is an –vector space of dimension less than or equal to .

The space of –polynomials is denoted by and, given a positive integer , the space of –polynomials of degree less than is denoted by

 L

Given positive integers with and an –tuple of –linearly independent elements of , the Gabidulin code is defined as

 Gk(a)def={(f(a1),…,f(an)) | f∈L

This code has a generator matrix of the form:

 ⎛⎜ ⎜ ⎜ ⎜ ⎜⎝a1…anaq1…aqn⋮⋮aqk−11…aqk−1n⎞⎟ ⎟ ⎟ ⎟ ⎟⎠.

Such codes are known to have minimum distance and to benefit from a decoding algorithm correcting up to half the minimum distance (see [9]).

The –tuple is referred to as the support of the code. Note that the support is not unique as shown by the following lemma which will be useful for our attack.

Let . Then

###### Proof

Let be a –polynomial of –degree and be the –polynomial of the same degree defined by . Then the codeword is equal to the codeword . This proves that and the converse inclusion is proved in a similar fashion.∎

### 1.3 The component-wise Frobenius map

In what follows, we will frequently apply the component-wise Frobenius map or its iterates to vectors or codes. Hence, we introduce the following notation. Given a vector and a nonnegative integer , we denote by the vector:

 v[s]def=(vqs1,…,vqsn).

Similarly, given a code and a positive integer , the code denotes the code

 C[s]def={c[s] | c∈C}.

### 1.4 Overbeck’s distinguisher

In [13], Overbeck proposes a general framework to break cryptosystems based on Gabidulin codes. The core of his attack is that a simple operation permits to distinguish Gabidulin codes from random ones. Indeed, given a random code of dimension , the expected dimension of the code equals and, equivalently is likely to be equal to . More generally, we have the following statement.

###### Proposition 2

If is a code of length and dimension chosen uniformly at random, then for a nonnegative integer and for a positive integer , we have

 P(dimFqm Crand+Crand[1]+⋯+Crand[s]⩽min(n,(s+1)k)−a)=O(q−ma).
###### Proof

See Appendix 0.A.∎

On the other hand, for a Gabidulin code, the behaviour with respect to such operations is completely different as explained in the following statement.

###### Proposition 3

Let be a word of rank , and be an integer. Then,

 Gk(a)∩Gk(a)[1] = Gk−1(a[1]); Gk(a)+⋯+Gk(a)[s] = Gk+s(a).
###### Example 1

As an illustration of the difference of behaviour of Gabidulin codes compared to random codes, given a Gabidulin code of length and dimension , then , while for a random code of the same length and dimension, tends to when tends to infinity.

## 2 Loidreau’s scheme

In order to mask the structure of Gabidulin codes and to resist to Overbeck’s attack, Loidreau suggested in [11] the following construction. Denote by a random generator matrix of a Gabidulin code . Fix an integer and an –vector subspace of of dimension . Let whose entries are all in . Then, let

 Gpubdef=GP−1.

We have the following encryption scheme.

Public key:

The pair where .

Secret key:

The pair .

Encryption:

Given a plaintext , choose a uniformly random vector of rank weight . The ciphertext is

 cdef=mGpub+e.
Decryption:

Compute,

 cP=mG+eP.

Since the entries of are all in then, the entries of are in the product space . The dimension of this space is bounded from above by . Therefore, using a classical decoding algorithm for Gabidulin codes, one can recover .

## 3 A distinguisher when λ=2

### 3.1 Context

The goal of this section is to establish a distinguisher for Loidreau’s cryptosystem instantiated with and a public code of dimension . Similarly to Overbeck’s attack, this distinguisher reposes on Propositions 2 and 3.

Similarly to the attacks of BBCRS system [3, 4], it is more convenient to work on the dual of the public code because of the following lemmas. By dual, we mean the orthogonal code with respect to the canonical inner product

###### Lemma 2 ([10, Page 52])

The code is a Gabidulin code for some of rank .

###### Lemma 3 ([4, Lemma 1])

Any full-rank generator matrix of can be decomposed as

 Hpub=HsecPT

where is a parity–check of the Gabidulin code .

The convenient aspect of the previous lemma is that the matrix has its entries in a small vector space, while its inverse has not.

### 3.2 The case λ=2

We suppose in this section that the vector space in which the matrix has all its entries has dimension :

 λ=dimFqV=2.

Note that, w.l.o.g, one can suppose that . Indeed, if is spanned over by , then one can replace by which spans the same code and has entries in and .

Thus, from now on, we suppose that for some . Consequently, can be decomposed as

 PT=P0+γP1,

where are square matrices with entries in . For convenience, we suppose from now on that and

are both invertible. Note that this actually holds with a high probability. If one of these matrices was not invertible, then the attack could probably be performed after minor adjustments.

We have seen that for some with . We define

 gdef=aP0andhdef=aP1.
###### Lemma 4

The code is spanned by

 g+γh,g[1]+γh[1],…,g[n−k−1]+γh[n−k−1].
###### Proof

For any . There exists such that

 c=P(a)PT=P(a)P0+γP(a)P1=P(g)+γP(h),

which yields the result. ∎

We can now state a crucial result.

###### Theorem 3.1

The dual of the public code satisfies:

###### Proof

Thanks to Lemma 4, we prove that is spanned by

 g+γh,⋯,g[n−k−1]+γh[n−k−1]andg[1]+γqh[1],…,g[n−k]+γqh[n−k].

Equivalently, is spanned by:

 g+γhandg[1],h[1],…,g[t],h[t]andg[n−k]+γqh[n−k].

Finally, a similar reasoning permits to show that is spanned by

 g+γhandg[1],h[1],…,g[n−k],h[n−k]andg[n−k+1]+γq2h[n−k+1],

and hence has dimension at most . ∎

As a conclusion, thanks to Proposition 2, we deduce that is distinguishable in polynomial time from a random code as soon as .

## 4 The attack

In this section, we derive an attack from the distinguisher defined in Section 3. In what follows, we suppose that and the public code has rate larger than so that the distinguisher introduced in Section 3 works on it. Recall that for some whose entries are –independent and is of the form for and . Finally recall that

 gdef=aP0andhdef=aP1.

In addition, we make the following assumptions:

1. ;

2. and is not contained in any subfield of ;

3. .

Assumption (1) has already been discussed in § 3.2. Assumption (2) is reasonable in order to prevent against possible attacks based on an exhaustive search of . Finally, Assumption (3) is what typically happens according to our experiments using Magma [2].

The aim of the attack is to recover the triple , or more precisely, to recover a triple such that

 C⊥pub=⟨g′[i]+γ′h′[i] | i=0,…,n−k−1⟩. (1)

Actually, the triple is far from being unique and any other triple satisfying (1) permits to decrypt messages (see further § 4.3). Let us describe an action of on such triples.

###### Proposition 4

Let such that and such that . Then, the triple satisfies (1).

###### Proof

It suffices to observe that for any ,

 g[i]+aδ+bcδ+dh[i]=1cδ+d((dg+bh)[i]+δ(cg+ah)[i]).

### 4.1 Step 1: using the distinguisher to compute some subcodes

As shown in the proof of Theorem 3.1, is spanned by:

 g+γhandg[1],h[1],…,g[r],h[r]andg[r+1]+γqh[r+1],

where . Then, by iterating intersections

 (C⊥pub+C⊥pub[1])∩(C⊥pub[1]+C⊥pub[2])∩⋯∩(C⊥pub[r]+C⊥pub[r+1]),

we obtain the code spanned by

 g[r]+γqrh[r]andg[r+1]+γqh[r+1].

Notice that Assumption (3) permits to prove that this intersection has exactly dimension . Applying the inverse of the –th Frobenius, we get the code spanned by

 g+γhandg[1]+γq1−rh[1].

Note that . Hence because of Assumption (2). Next, one can compute

 C⊥pub∩⟨g+γh,g[1]+γq1−rh[1]⟩=⟨g+γh⟩ (2)

and

 ⟨g+γh,g[1]+γq1−rh[1]⟩ + ⟨g+γh⟩[1] = ⟨g+γh,g[1]+γq1−rh[1],g[1]+γqh[1]⟩ = ⟨g+γh,g[1],h[1]⟩.

Similarly, we compute the intersection with and get

 C⊥pub[−1]∩⟨g+γh,g[1],h[1]⟩=⟨g[1]+γqm−1h[1]⟩. (3)

Applying the inverse Frobenius to the last code, we get . Since, from (2), we also know , one can compute

 ⟨g+γh⟩+⟨g[1]+γqm−1h[1]⟩[−1]=⟨g+γh,g+γqm−2h⟩=⟨g,h⟩. (4)

Next, for any one can compute

 Cpub⊥∩⟨g,h⟩[i]=⟨g[i]+γh[i]⟩.

By applying the –th inverse Frobenius to the previous result, we obtain the space for any . In summary, we know the spaces

 ⟨g+γh⟩,⟨g+γq−1h⟩,…,⟨g+γq−rh⟩.

In addition, from Lemma 1, the vector is determined up to some multiplicative constant. Therefore, one can choose an arbitrary element of and suppose that this element is .

### 4.2 Step 2. Finding γ

In summary, the vector and the spaces for any are known. To compute , we will use the following lemma.

###### Lemma 5

For , , there exists a unique pair such that .

###### Proof

It suffices to observe that . ∎

The pairs of vectors can be easily computed. Thus, from now on, we suppose we know them. In addition, despite , and are unknown, a calculation permits to show that have the following expressions.

 uij=γq−j−γγq−j−γq−i⋅(g+γq−ih)andvij=γ−γq−iγq−j−γq−i⋅(g+γq−jh). (5)

Consider the vectors and . They are collinear since, from (5), they are both multiples of . Therefore, one can compute the scalar such that . From (5) we deduce that satisfies the following relation.

 γq−2−γγq−2−γq−1=α⋅γq−2−γγq−2−γq−1⋅ (6)

Or equivalently, is a root of the polynomial

 Qγ(X)def=(Xq−Xq3)(X−Xq2)−αq3(X−Xq3)(Xq−Xq2).

One can easily check that divides and we set

 Pγ(X)def=Qγ(Xq−X)q+1⋅

The element we look for is a root of but actually, the forthcoming Proposition 5 provides the description of the other roots. We first need a technical Lemma.

###### Lemma 6

Let , let be two nonnegative integers and set . Then,

###### Proposition 5

The set of roots of equals the orbit of under the action of . Equivalently, any root of is of the form for such that .

###### Proof

First, notice that and hence

 degPγ=degQγ−q(q+1)=q3−q=|PGL(2,Fq)|.

Second, for any , Lemma 6 entails

 Pγ(aX+bcX+d)=1(cX+d)q3−q⋅Pγ(X).

Since , then and hence, .

We proved that any element in the orbit of under is a root of . To conclude, we need to prove that the orbit of under has cardinality which means that the stabilizer of with respect to this group action is trivial. Indeed, suppose that

 γ=aγ+bcγ+d,

for some . Then is a root of , which has degree at most . This contradicts Assumption (2).∎

Thanks to Propositions 4 and 5, we deduce that choosing an arbitrary root of provides a candidate for and there remains to compute providing our triple. Since for some , then,

 g+γh=(g′+γ′h′)

where

 g′def=1(cγ′+d)⋅(dg+bh)andh′def=1(cγ′+d)⋅(cg+ah)

Considering (5) and using Lemma 6, we get

 u12=γ′q−2−γ′γ′q−2−γ′q−1⋅(g′+γ′q−1h′).

Consequently, we know and the vectors and . Thus, we can also compute

 g′+γ′q−1h′=γ′q−2−γ′q−1γ′−2−γ′u12.

Knowing , and permits to recover .

### 4.3 End of the attack

Given the pair , compute the unique matrix such that . Then,

 C⊥pub=Gn−k(g′)⋅(In+γ′Q)

and this representation of the dual provides all the elements necessary to decode, that is to decrypt any ciphertext.

## Conclusion

We provided a distinguisher à la Overbeck for the public keys of Loidreau’s scheme when and the public code has rate . From this distinguisher, we are able to derive a polynomial time key recovery attack. This attack can probably be extended to other values of when the public code rate satisfies . Therefore, such parameters should be avoided in Loidreau’s scheme.

## 5 Acknowledgements

The second author was funded by French ANR projects ANR-15-CE39-0013 Manta and ANR-17-CE39-0007 CBCrypt.

## References

• [1] Baldi, M., Bianchi, M., Chiaraluce, F., Rosenthal, J., Schipani, D.: Enhanced public key security for the McEliece cryptosystem. J. Cryptology 29(1), 1–27 (2016). https://doi.org/10.1007/s00145-014-9187-8, http://dx.doi.org/10.1007/s00145-014-9187-8
• [2] Bosma, W., Cannon, J., Playoust, C.: The Magma algebra system I: The user language. J. Symbolic Comput. 24(3/4), 235–265 (1997)
• [3] Couvreur, A., Gaborit, P., Gauthier-Umaña, V., Otmani, A., Tillich, J.P.: Distinguisher-based attacks on public-key cryptosystems using Reed-Solomon codes. Des. Codes Cryptogr. 73(2), 641–666 (2014)
• [4] Couvreur, A., Otmani, A., Tillich, J.P., Gauthier-Umaña, V.: A polynomial-time attack on the BBCRS scheme. In: Katz, J. (ed.) Public-Key Cryptography - PKC 2015. LNCS, vol. 9020, pp. 175–193. Springer (2015)
• [5] Delsarte, P.: Bilinear forms over a finite field, with applications to coding theory. J. Comb. Theory, Ser. A 25(3), 226–241 (1978)
• [6] Gabidulin, E.M.: Theory of codes with maximum rank distance. Problemy Peredachi Informatsii 21(1), 3–16 (1985)
• [7] Gabidulin, E.M., Paramonov, A.V., Tretjakov, O.V.: Ideals over a non-commutative ring and their applications to cryptography. In: Advances in Cryptology - EUROCRYPT’91. pp. 482–489. No. 547 in LNCS, Brighton (Apr 1991)
• [8] Gaborit, P., Murat, G., Ruatta, O., Zémor, G.: Low rank parity check codes and their application to cryptography. In: Proceedings of the Workshop on Coding and Cryptography WCC’2013. Bergen, Norway (2013), available on www.selmer.uib.no/WCC2013/pdfs/Gaborit.pdf
• [9] Loidreau, P.: A Welch–Berlekamp like algorithm for decoding Gabidulin codes. In: Ytrehus, Ø. (ed.) Coding and Cryptography. pp. 36–45. Springer Berlin Heidelberg, Berlin, Heidelberg (2006)
• [10] Loidreau, P.: Rank metric and cryptography. Accreditation to supervise research, Université Pierre et Marie Curie - Paris VI (Jan 2007), https://tel.archives-ouvertes.fr/tel-00200407
• [11] Loidreau, P.: A new rank metric codes based encryption scheme. In: International Workshop on Post-Quantum Cryptography. pp. 3–17. Springer (2017)
• [12] McEliece, R.J.: A Public-Key System Based on Algebraic Coding Theory, pp. 114–116. Jet Propulsion Lab (1978), dSN Progress Report 44
• [13] Overbeck, R.: Structural attacks for public key cryptosystems based on Gabidulin codes. J. Cryptology 21(2), 280–301 (2008)

## Appendix 0.A Proof of Proposition 2

### 0.a.1 Preliminaries on Gaussian binomial coefficients

###### Notation 1

In what follows, we denote by the Gaussian binomial coefficient representing the number of subspaces of dimension of a vector space of dimension over .

###### Lemma 7

There exists a positive constant such that for any pair of positive integers such that , we have

 qk(n−k)⩽[nk]q⩽C⋅qk(n−k).
###### Proof

By definition of Gaussian binomials, we have

 [nk]q=k−1∏t=0qn−qtqk−qt=qk(n−k)k−1∏t=01−qt−n1−qt−k⋅

Since , we get

 k−1∏t=01−qt−n1−qt−k⩾1,

which yields the left hand inequality. To get the other equality, we need to bound from above the product:

 k−1∏t=01−qt−n1−qt−k⩽k−1∏t=011−qt−k=k−1∏j=011−1qj+1,

where the last equality is obtained by applying the change of variables . Set

 akdef=k−1∏j=011−1qj+1⋅

The sequence is increasing and converges. Indeed,

 log(ak)=k−1∑j=0−log(1−1qj+1)

and the series with general term converges. As a conclusion, the right-hand inequality is obtained by taking

 Cdef=∞∏j=011−1qj+1⋅

###### Remark 2

A finer analysis would permit to prove that . In particular, since , we have that .

### 0.a.2 The proof

Let be a subspace of chosen uniformly at random among its subspaces of dimension . From we build the map

 Ψ:⎧⎪ ⎪⎨⎪ ⎪⎩(s+1) timesCrand×⋯×Crand⟶Fnqm(c0,…,cs)⟼c0+c[1]1+⋯+c[s]s.

The image of this map is and hence the dimension of is related to the dimension of the kernel of

. Therefore, our approach will consist in estimating

.

We have

 E(|kerΨ|)=∑(x0,…,xs)∈(Fnqm)sx0+x[1]1+⋯+x[s]s=0P(x0,…,xs∈Crand). (7)
###### Lemma 8

Let be a subspace of of dimension . Then

 P(A⊆Crand)⩽C⋅q−mt(n−k),

where is the constant of Lemma 7.

###### Proof

We have

 P(A⊆Crand)=[n−tk−t]qm[nk]qm⋅

Using Lemma 7 we get the upper bound,

 P(A⊆Crand)⩽C⋅qm(k−t)(n−k)⋅q−mk(n−k)=C⋅q−mt(n−k).

For any , we introduce the set

 Etdef={(x0,…,xs)∈(Fnqm)s ∣∣∣ x0+x[1]1+⋯+x[s]s=0dimFqm⟨x0,…,xs⟩=t}⋅

Thanks to (7) and Lemma 8, we can write that

 E(|kerΨ|)⩽C⋅s+1∑t=0q−mt(n−k)|Et|. (8)
###### Lemma 9

Let . Then,

 |Et|⩽C⋅q(