Uniform Minors in Maximally Recoverable Codes

06/06/2019 ∙ by Matthias Grezet, et al. ∙ Mälardalens högskola 0

In this letter, locally recoverable codes with maximal recoverability are studied with a focus on identifying the MDS codes resulting from puncturing and shortening. By using matroid theory and the relation between MDS codes and uniform minors, the list of all the possible uniform minors is derived. This list is used to improve the known non-asymptotic lower bound on the required field size of a maximally recoverable code.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

With the exponential growth of data needed to be stored remotely, distributed storage systems (DSSs) using erasure-correcting codes have become attractive due to their high reliability and low storage overhead. A class of codes called locally recoverable codes (LRCs) has been introduced in [1, 2] as an alternative to traditional maximum distance separable (MDS) codes to improve node repair efficiency by allowing one failed node to be repaired by only accessing a few other nodes.

A linear -LRC is a linear code of length and dimension over such that every codeword symbol is contained in a repair set with and the minimum Hamming distance of the restriction of the code to is at least . In other words, any symbol can be determined by the values of at most other symbols. Constructions of distance-optimal LRCs with field size of order have been given in [3].

LRCs with maximal recoverability (MR-LRCs) or maximally recoverable codes (also known as partial MDS or PMDS) have been introduced in [4]. MR-LRCs are a subclass of distance-optimal LRCs that can correct any erasure pattern that is information-theoretically correctable. Formally, an MR-LRC is an -LRC whose codeword symbols are partitioned into disjoint repair sets and any set of symbols with is an information set. The number of heavy (global) parity checks is . This definition can be extended to allow the repair sets to correct erasures but for the clarity of this letter, we will only consider .

MR-LRCs drew a lot of attention recently with many papers being devoted to the construction of general classes of MR-LRCs over the lowest possible field size. While a field size linear in is sufficient for optimal LRCs, known constructions of MR-LRCs for any parameters are generally exponential in or . A general construction for local erasures with field size of order was obtained in [5]. The best constructions so far for MR-LRCs tolerating local erasures were given in [6, 7], where [6] obtained field sizes of order and , and [7] obtained a field size of order .

However, little is known regarding the lower bound on the required field size . In [5], the authors proved that by puncturing one element per repair set, the resulting code is an MDS code and therefore . Recently, [8] gave the first asymptotic superlinear lower bound for MR-LRCs tolerating erasures when is constant and may grow with . The bound is the following:

In this letter, we pursue the approach started by [5] and identify, for each dimension, the largest length of an MDS code obtained by puncturing and shortening. Our main tools to achieve this come from matroid theory. The link between MR-LRCs and matroids was already used in [9] where the authors computed the Tutte polynomial of MR-LRCs to derive the weight enumerator and higher support weights. Here, we work with the collection of flats and matroid minors to construct the largest possible uniform minors in MR-LRCs and thus, the largest MDS codes. These minors are then used to improve the non-asymptotic lower bound found in [5], both with and without assuming the MDS conjecture.

Ii Preliminaries

We denote the set by and the set of all subsets of by . A generator matrix of a linear code is where

is a column vector for

. Matroids have many equivalent definitions in the literature. Here, we choose to define matroids via their rank functions. Much of the contents in this section can be found in more detail in [10].

Definition 1.

A (finite) matroid is a finite set together with a rank function such that for all subsets ,

There is a unique matroid associated to a linear code where and is the dimension of the restriction of to for .

Two matroids and are isomorphic if there exists a bijection such that for all subsets . We denote two isomorphic matroids by .

Let be a matroid. The closure operator is defined by . A subset is a flat if and the collection of flats is denoted by .

Definition 2.

The uniform matroid is a matroid with a ground set and a rank function for . In particular, the flats are .

The following straightforward observation gives a characterization of MDS codes.

Proposition 1.

A linear code is an -MDS code of length and dimension if and only if is the uniform matroid .

There are several elementary operations that are useful for explicit constructions of matroids, as well as for analyzing their structure.

Definition 3.

Let be a matroid and . Then

  1. The restriction of to is the matroid , where for .

  2. The contraction of by is the matroid , where for .

  3. For , a minor of is the matroid obtained from by restriction to and contraction by . Observe that this does not depend on the order in which the restriction and contraction are performed.

The deletion of by , denoted by , is the restriction of to . These operations can be equivalently defined via the generator matrix of a code of length . If we label the columns of from to , then the restriction to is the same as considering the submatrix formed by the columns indexed in and the contraction by is the projection from the columns indexed in . Thus, the deletion and contraction correspond to puncturing and shortening of codes, respectively. We can also describe the flats of a minor.

Proposition 2.

Let be a matroid and with , then

  1. ,

  2. .

Iii Uniform minors and a lower bound on

As mentioned in the introduction, this letter pursues two objectives: classifying the uniform minors or MDS codes inside an MR-LRC and improving the lower bound on the required field size. The second problem is highly related to the MDS conjecture.

Conjecture 1 ([11]).

If then a linear -MDS code over has length unless and or , in which case .

The conjecture is proven when is a prime or when for in [12]. Without assuming the MDS conjecture, the following lemma bounds the field size.

Lemma 1 ([12] Lemma 1.2).

Any -MDS code over satisfies .

Regarding the classification of the uniform minors, we first give the structure of the flats of the associated matroid to an MR-LRC. For simplicity, if is the matroid associated to an MR-LRC, then is called an -MR matroid.

Proposition 3.

Let be an -MR matroid. Then the flats are

The rank of a flat is .

Proof.

A set with is not a flat if and only if there exists a repair set such that . Indeed, if then and therefore . Moreover, the rank function of is given by . ∎

The following theorem by Higgs is known as the Scum Theorem. It significantly restricts the sets that one must consider in order to find all minors of as .

Theorem 1 ([13] Proposition 3.3.7).

Let be a minor of a matroid . Then there is a pair of sets with and , such that . Further, if for all , we have , then can be chosen to be a flat of .

The next four propositions classify all the uniform minors in an MR-LRC. One uniform minor has already been obtained in [5] by deleting one element per repair set. It can be formulated as follows.

Proposition 4 ([5] Theorem 19).

Let be an -MR matroid. Then contains a minor where

(1)
Proposition 5.

Let be an -MR matroid. Then contains a minor where

(2)
Proof.

By Theorem 1, we are looking for a flat such that and for all with we have . Then, by Proposition 2, . When , we will also use an extra deletion.

The second condition implies that if there exists a non-empty with , then either or we need to delete an element from . The reason is that if , then since is a flat. If , then let and choose . We have but is not a flat because .

Let us first assume that . Because of the previous argument, we need . Then, for all with , we have . Thus, is a uniform minor with rank and size

Assume now that . In this case, we need to add a part of a repair set to complete the rank and delete an element to remove the unwanted flat. Let . Then, we have

Since the rank of the union of all repair sets is , there exists an extra repair set . Let such that . Notice that . Now let . Then, and

Furthermore, for all , we have . It remains to delete one element in to get rid of the flat . To this end, let . We have and . Hence is a uniform minor where

Proposition 6.

Let be an -MR matroid and . Then contains a minor where

(3)
Proof.

By Theorem 1, we are looking for a flat such that and for all with we have .

The second condition implies that if and , then . Otherwise, if , we have . Then, let and choose . We have and . Therefore, is not a flat since . To construct , we distinguish two cases depending on the number of repair sets .

Assume first that . Then, let where with . Since is an MR-matroid, we have that . Let also such that and define .

Hence, and for all with , we have since consists of independent elements where no more than elements of are contained in the same repair set.

Assume now that . This means that there are not enough repair sets to build an independent set as in the previous case and has to contain some . Thus, we are looking for the minimum number of repair sets that has to contain before we can add an independent set. Formally, is given by

The condition on simplifies as follows.

Therefore, we have that . Now, let and where and with such that . Then, the rank of is . By definition of , we have . Then, let and such that . Notice that . Finally, define . We have indeed that and . Moreover, by the same argument as in the previous case, for all with , we have .

Hence, is a uniform minor with rank and size .

We are left with the case . In fact, requesting forces the deletion of one element per repair set and the minor obtained is a subminor of the uniform minor obtained in Proposition 4. We state it here for completeness.

Proposition 7.

Let be an -MR matroid and . Then contains a minor where

(4)
Proof.

We want and such that . Since , it means that for all with , we have . In particular, sets of size should also be flats. Therefore, we cannot have and one element needs to be deleted from or be contained in . Since the two options yield the same size , we can choose to delete them first. Let with and let . As in Proposition 4, we have . Let with . Hence with . ∎

The techniques developed here easily generalize to the case when by taking the size of a repair set to be and deleting elements instead of .

When assuming the MDS conjecture, only the code length matters in the lower bound on the field size. Therefore, assuming the MDS conjecture, the bound on the field size of an MR-LRC is the largest size of a uniform minor minus one except on some special cases when is even. The next theorem gives the largest size of all the uniform minors found in the previous propositions. As such, it does not depend on the MDS conjecture.

Theorem 2.

Let be an -MR matroid with . The largest size of a uniform minor is

Proof.

We compare the sizes obtained in (1), (2), and (3). The case is straightforward as it is the largest size between (1) and (2). Assume now that . Let and where and .

First notice that both and are upper bounded by since . Thus, if , then and .

Suppose now that and let . Then, we have

Hence, and . We also have that for all . Since we already saw that and by the assumption on , we have that , this implies that is the maximum size when . ∎

Figure 1 displays the comparison between the length (1), (2), and (3) for fixed and . As we can see, (1) is the largest length in the high-rate regime while (3) is the largest length in the low-rate regime. While high-rate codes are preferable for storage, low-rate codes have advantages in terms of availability of hot data and lead to better rates when considering private information retrieval schemes.

Fig. 1: Comparison between the sizes (1), (2), and (3) when grows, , and .

Without assuming the MDS conjecture, we can use Lemma 1 to obtain a lower bound on the field size. When applying the lemma to the uniform minor obtained in Proposition 6, the dimension cancels out and the bound is maximized when .

Theorem 3.

Any MR-LRC over satisfies

where .

Even if these bounds are still far from the asymptotic bound in [8], they improve the non-asymptotic bound in [5], which is , for low-rate MR-LRCs. Indeed, a necessary condition for the new bounds to be better is that . More precisely, the new bound for improves on when . The bound when improves on when for ; for ; and directly when for all .

Iv Conclusion

In this letter, we studied maximally recoverable codes with a focus on classifying their uniform minors. As a direct consequence, we obtained the largest length of an MDS code inside an MR-LRC. Using the relation between MDS codes and the field size, we derived a lower bound on the required field size of an MR-LRC improving on the non-asymptotic bound in the low-rate regime. However, the gap between the lower bounds and the constructions remains an intriguing open problem. In particular, our results show that new techniques not relying on the MDS conjecture need to be found in order to close it.

References

  • [1] P. Gopalan, C. Huang, H. Simitci, and S. Yekhanin, “On the locality of codeword symbols,” IEEE Trans. Inf. Theory, vol. 58, no. 11, pp. 6925–6934, Nov. 2012.
  • [2] D. S. Papailiopoulos and A. G. Dimakis, “Locally repairable codes,” in Proc. IEEE Int. Symp. Inf. Theory, Cambridge, MA, Jul. 2012, pp. 2771–2775.
  • [3] I. Tamo and A. Barg, “A family of optimal locally recoverable codes,” IEEE Trans. Inf. Theory, vol. 60, no. 8, pp. 4661–4676, Aug. 2014.
  • [4] M. Chen, C. Huang, and J. Li, “On the maximally recoverable property for multi-protection group codes,” in Proc. IEEE Int. Symp. Inf. Theory, Nice, France, Jun. 2007, pp. 486–490.
  • [5] P. Gopalan, C. Huang, B. Jenkins, and S. Yekhanin, “Explicit maximally recoverable codes with locality,” IEEE Trans. Inf. Theory, vol. 60, no. 9, pp. 5245–5256, Jun. 2014.
  • [6] R. Gabrys, E. Yaakobi, M. Blaum, and P. H. Siegel, “Constructions of partial MDS codes over small fields,” IEEE Trans. Inf. Theory, Jun. 2018.
  • [7] U. Martínez-Peñas and F. R. Kschischang, “Universal and dynamic locally repairable codes with maximal recoverability via sum-rank codes,” in Proc. Annu. Allerton Conf. Commun., Control, and Comput., Monticello, IL, Feb. 2018, pp. 792–799.
  • [8] S. Gopi, V. Guruswami, and S. Yekhanin, “Maximally recoverable LRCs: A field size lower bound and constructions for few heavy parities,” in Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms, San Diego, CA, Jan. 2019, pp. 2154–2170.
  • [9] V. Lalitha and S. V. Lokam, “Weight enumerators and higher support weights of maximally recoverable codes,” in Proc. Annu. Allerton Conf. Commun., Control, and Comput., Monticello, IL, Apr. 2015, pp. 835–842.
  • [10] R. Freij-Hollanti, C. Hollanti, and T. Westerbäck, “Matroid theory and storage codes: bounds and constructions,” in Network Coding and Subspace Designs.   Springer, Jan. 2018, pp. 385–425.
  • [11] B. Segre, “Curve razionali normali ek-archi negli spazi finiti,” Annali di Matematica Pura ed Applicata, vol. 39, no. 1, pp. 357–379, Dec. 1955.
  • [12] S. Ball, “On sets of vectors of a finite vector space in which every subset of basis size is a basis,” Journal of the European Mathematical Society, vol. 14, no. 3, pp. 733–748, Mar. 2012.
  • [13] H. H. Crapo and G.-C. Rota, On the foundations of combinatorial theory: Combinatorial geometries.   MIT press Cambridge, Mass., 1970.