Barriers for fast matrix multiplication from irreversibility

1. Introduction

Determining the asymptotic algebraic complexity of matrix multiplication is a central open problem in algebraic complexity theory. Recent “barrier results” show various limitations for certain approaches to yield fast matrix multiplication algorithms [AFLG15, BCC $^{+}$ 17a, BCC $^{+}$ 17b, AW18a, AW18b]. Constructions of fast matrix multiplication algorithms typically (and this is in particular true for the most successful approaches) consist of two components: an efficient reduction of matrix multiplication to an intermediate problem and an efficient algorithm for the intermediate problem. We introduce a general barrier for such constructions based on a new notion called irreversibility, providing stronger limitations than in previous work. The barrier we introduce in this paper is build on the framework of Strassen developed in [Str87, Str88, Str91, CVZ18].

The asymptotic algebraic complexity of matrix multiplication is succinctly represented by the matrix multiplication exponent $ω$ , which is the infimum over all real numbers $β$ such that $n \times n$ matrices can be multiplied with $O (n^{β})$ algebraic operations. The state-of-the-art is $2 \leq ω \leq 2.37..$ and there has been tremendous effort to obtain better upper bounds on $ω$ [CW90, Sto10, Wil12, LG14, CU03, CU13].

An intuitive explanation of our barrier is as follows. In the language of tensors, the matrix multiplication exponent $ω$ is the optimal “rate of transformation” from the “unit tensor” to the “matrix multiplication tensor”,

(1)

unit tensor ω - \to matrix % multiplication tensor .

The rate of transformation naturally satisfies a triangle inequality and thus upper bounds on $ω$ can be obtained by combining the rate of transformation $α_{1}$ from the unit tensor to some intermediate tensor and the rate of transformation $α_{2}$ from the intermediate tensor to the matrix multiplication tensor; this is the two-component approach alluded to earlier,

(2)

unit tensor α_{1} - - \to % intermediate tensor α_{2} - - \to matrix % multiplication tensor .

We define the irreversibility of the intermediate tensor as the necessary “loss” that will occur when transforming the unit tensor to the intermediate tensor followed by transforming the intermediate tensor back to the unit tensor. It is well-know that the transformation rate from the matrix multiplication tensor to the unit tensor is $\frac{1}{2}$ , so we can extend (2) to

(3)

unit tensor α_{1} - - \to % intermediate tensor α_{2} - - \to matrix % multiplication tensor 1 / 2 - - \to unit tensor .

We thus see that $α_{1} α_{2}$ is directly related to the irreversibility of the intermediate tensor, and hence the irreversibility of the intermediate tensor provides limitations on the upper bounds on $ω$ that can be obtained from (2). In particular, any fixed irreversible intermediate tensor cannot show $ω = 2$ via (2), since the matrix multiplication tensor is reversible when $ω = 2$ .

To exemplify our barrier we show that the support functionals [Str91] and quantum functionals [CVZ18] give (so far, the best) lower bounds on the irreversibility of the following families of tensors:

[leftmargin=2em]
the small Coppersmith–Winograd tensors

${c w}_{q} = q \sum i = 1 e_{0, i, i} + e_{i, 0, i} + e_{i, i, 0}$
the big Coppersmith–Winograd tensors

${C W}_{q} = e_{0, 0, q + 1} + e_{0, q + 1, 0} + e_{q + 1, 0, 0} + q \sum i = 1 e_{0, i, i} + e_{i, 0, i} + e_{i, i, 0}$
the reduced polynomial multiplication tensors

$t_{n} = n - 1 \sum \begin{matrix} i, j, k = 0 : i + j = k \end{matrix} e_{i, j, k}$

which for small parameters leads to the following explicit barriers:

q

{c w}_{q}

-barrier 2 2 3 2.02.. 4 2.06.. 5 2.09.. 6 2.12.. 7 2.15..

q

{C W}_{q}

-barrier 1 2.16.. 2 2.17.. 3 2.19.. 4 2.20.. 5 2.21.. 6 2.23..

n

t_{n}

-barrier 1 2.17.. 2 2.16.. 3 2.15.. 4 2.15.. 5 2.14.. 6 2.14..

Indeed, as suggested by the values in the above tables, the ${c w}_{q}$ -barrier and ${C W}_{q}$ -barrier increase with $q$ (converging to 3), whereas the $t_{n}$ -barrier decreases with $n$ (converging to 2).

Compared to Ambainis, Filmus and Le Gall [AFLG15] our barriers are valid for a larger class of approaches (and naturally we obtain lower barriers). Compared to Alman and Williams [AW18b] our barriers are valid for a larger class of approaches but our barriers are also higher. As a variation on our barrier we introduce a “monomial” version. Compared to Blasiak, Church, Cohn, Grochow, Naslund, Sawin and Umans [BCC $^{+}$ 17a], and Blasiak, Church, Cohn, Grochow and Umans [BCC $^{+}$ 17b] our monomial barriers are valid for a larger class of approaches. We have not tried to optimise the barriers that we obtain here, but focus instead on introducing the barrier itself.

It will become clear to the reader during the development of our ideas that they not only apply to the problem of fast matrix multiplication, but extend to give barriers for the more general problem of constructing fast rectangular matrix multiplication algorithms or even transformations between arbitrary powers of tensors. Such transformations may represent, for example, asymptotic slocc (stochastic local operations and classical communication) reductions among multipartite quantum states [BPR $^{+}$ 00, DVC00, VDDMV02, HHHH09].

We define irreversibility in Section 2. In Section 3 we introduce the irreversibility barrier. Finally, in Section 4 we present explicit irreversibility barriers.

2. Irreversibility

We begin by introducing some standard notation and terminology. Then we discuss a useful notion called the relative exponent and we define the irreversibility of a tensor. After that we introduce the monomial versions of these ideas and discuss so-called balanced tensors.

2.1. Standard definitions

We assume familiarity with tensors and with the tensor Kronecker product and direct sum. All our tensors will be 3-tensors over some fixed but arbitrary field $F$ . For two tensors $t \in F^{n_{1}} \otimes F^{n_{2}} \otimes F^{n_{3}}$ and $s \in F^{m_{1}} \otimes F^{m_{2}} \otimes F^{m_{3}}$ we write $t \geq s$ and say $t$ restricts to $s$ if there are linear maps $A_{i} : F^{n_{i}} \to F^{m_{i}}$ such that $(A_{1}, A_{2}, A_{3}) \cdot t = s$ . For $n \in N$ we define the diagonal tensor (also called the rank- $n$ unit tensor) $⟨n⟩\coloneqq∑ni=1ei,i,i∈Fn⊗Fn⊗Fn$ . The tensor rank of $t$ is defined as $R(t)\coloneqqmin{n∈N:t≤⟨n⟩}$ (this coincides with the definition that $R (t)$ is the smallest size of any decomposition of $t$ into a sum of simple tensors) and the subrank of $t$ is defined as $Q(t)\coloneqqmax{n∈N:⟨n⟩≤t}$ . The asymptotic rank of $t$ is defined as

(4)

\lx@converttounder\wtildeR(t)\coloneqqlimn→∞R(t⊗n)1/n=infnR(t⊗n)1/n

and the asymptotic subrank of $t$ is defined as

(5)

\lx@converttounder\wtildeQ(t)\coloneqqlimn→∞Q(t⊗n)1/n=supnQ(t⊗n)1/n.

The above limits exist and equal the respective infimum and supremum by Fekete’s lemma. For $a, b, c \in N_{\geq 1}$ the matrix multiplication tensor $⟨ a, b, c ⟩$ is defined as

(6)

⟨a,b,c⟩\coloneqqa∑i=1b∑j=1c∑k=1e(i,j),(j,k),(k,i)∈(Fa⊗Fb)⊗(Fb⊗Fc)⊗(Fc⊗Fa).

The matrix multiplication exponent is defined as $ω\coloneqqlog2\lx@converttounder\wtildeR(⟨2,2,2⟩)$ . The meaning of $ω$ in terms of algorithms is: for any $ε > 0$ there is an algorithm that for any $n \in N$ multiplies two $n \times n$ matrices using $O (n^{ω + ε})$ scalar additions and multiplications. The difficulty of determining the asymptotic rank of $⟨ 2, 2, 2 ⟩$ is to be contrasted with the situation for the asymptotic subrank; to put it in Strassen’s words: Unlike the cynic, who according to Oscar Wilde knows the price of everything and the value of nothing, we can determine the asymptotic value of $⟨ h, h, h ⟩$ precisely [Str88],

(7)

\lx@converttounder\wtildeQ(⟨h,h,h⟩)=h2.

2.2. Relative exponent

For a clean exposition of our barrier we will use the notion of relative exponent, which we will define in this section. This notion is inspired by the notion of rate from information theory and alternatively can be seen as a versatile version of the notion of the asymptotic preorder for tensors of Strassen. In the context of tensors, the relative exponent previously appeared in [VC15].

Assumption 1.

To avoid irrelevant technicalities, we will from now on, without further mentioning, only consider tensors that are not of the form $u \otimes v \otimes w$ .

Definition 2.

For two tensors $t \in F^{n_{1}} \otimes F^{n_{2}} \otimes F^{n_{3}}$ and $s \in F^{m_{1}} \otimes F^{m_{2}} \otimes F^{m_{3}}$ we define the relative exponent from $t$ to $s$ as

(8)		$ω (t, s)$	$\coloneqqlimn→∞1nmin{m∈N:t⊗m≥s⊗n}$
(9)			$= sup n \frac{1}{n} min {m \in N : t^{\otimes m} \geq s^{\otimes n}} .$

The limit is a supremum by Fekete’s lemma. Let us briefly relate the relative exponent to the basic notions and results stated earlier. The reader verifies directly that the identities

(10)		$ω(⟨2⟩,t)=log2\lx@converttounder\wtildeR(t)$
(11)		$ω(t,⟨2⟩)=1/(log2\lx@converttounder\wtildeQ(t))$

hold. By definition of the matrix multiplication exponent $ω$ holds

(12)

ω (⟨ 2 ⟩, ⟨ 2, 2, 2 ⟩) = ω .

We know from (7) that

(13)

ω (⟨ 2, 2, 2 ⟩, ⟨ 2 ⟩) = \frac{1}{2} .

The relative exponent has the following two basic properties that the reader verifies directly.

Proposition 3.

Let $s$ , $t$ and $u$ be tensors.

[label=()]
$ω (t, t) = 1$ .
$ω (s, t) ω (t, u) \geq ω (s, u)$ (triangle inequality).

2.3. Irreversibility.

Our barrier framework relies crucially on the irreversibility of a tensor, a new notion that we define now.

Definition 4.

We define the irreversibility of a tensor $t$ as the product of the relative exponent from $⟨ 2 ⟩$ to $t$ and the relative exponent from $t$ to $⟨ 2 ⟩$ , i.e.

(14)

i(t)\coloneqqω(⟨2⟩,t)ω(t,⟨2⟩).

Thus $i (t)$ measures the extent to which the asymptotic conversion from $⟨ 2 ⟩$ to $t$ is irreversible, explaining the name. Equivalently, the irreversibility is the ratio of the logarithms of the asymptotic rank and the asymptotic subrank, i.e.

(15)

i(t)=log2\lx@converttounder\wtildeR(t)log2\lx@converttounder\wtildeQ(t).

From the basic properties of the relative exponent (Proposition 3) follows directly the inequality $i (t) = ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩) \geq ω (⟨ 2 ⟩, ⟨ 2 ⟩) = 1$ .

Proposition 5.

For any tensor $t$ holds that

(16)

i (t) \geq 1.

Definition 6.

Let $t$ be a tensor.

If $i (t) = 1$ , then we say that $t$ is reversible.
If $i (t) > 1$ , then we say that $t$ is irreversible.

For example, for any $n \in N$ the diagonal tensor $⟨ n ⟩ = \sum_{i = 1}^{n} e_{i, i, i}$ is reversible. In fact, we do not know of any other reversible tensors.

For the matrix multiplication tensor $⟨ 2, 2, 2 ⟩$ we have $2 i (⟨ 2, 2, 2 ⟩) = ω$ (using (13)). Thus if $ω = 2$ , then $⟨ 2, 2, 2 ⟩$ is reversible (and also any other $⟨ n, n, n ⟩$ ). As we will see in Section 3, this is ultimately the source of our barrier.

Irreversible tensors exist. For example, $W = e_{0, 0, 1} + e_{0, 1, 0} + e_{1, 0, 0}$ is irreversible. Namely, it is well-known that $log2\lx@converttounder\wtildeR(W)=1$ and that $log2\lx@converttounder\wtildeQ(W)=h(1/3)=0.918..$ [Str91, Theorem 6.7], so $i (W) = 1.088.. > 1$ . In Section 4 we will compute lower bounds on the irreversibility of the small and big Coppersmith–Winograd tensors (that play a crucial role in the best upper bounds on $ω$ ).

2.4. Monomial relative exponent and monomial irreversibility

The following restrained version of relative exponent and irreversibility will be relevant. For two tensors $t \in F^{n_{1}} \otimes F^{n_{2}} \otimes F^{n_{3}}$ and $s \in F^{m_{1}} \otimes F^{m_{2}} \otimes F^{m_{3}}$ we write $t \geq_{M} s$ and say $t$ monomially restricts to $s$ if there are linear maps $A_{i} : F^{n_{i}} \to F^{m_{i}}$ , the corresponding matrices of which are generalised sub-permutation matrices in the standard basis, such that $(A_{1}, A_{2}, A_{3}) \cdot t = s$ [Str87, Section 6]. Replacing the preorder $\geq$ by $\geq_{M}$ in Section 2 gives the notions of monomial subrank $Q_{M}$ , monomial asymptotic subrank $\lx@converttounder\wtildeQM$ and monomial relative exponent $ω_{M}$ . (For simplicity we will use monomial restriction here, but our results will also hold with $\geq_{M}$ replaced by monomial degeneration $⊵_{M}$ defined in [Str87, Section 6].) Note that the notions $Q_{M}$ and $\lx@converttounder\wtildeQM$ only depend on the support of the tensor, and not on the particular values of the nonzero coefficients. We define the monomial irreversibility $i_{M} (t)$ of $t$ as the product of the (normal) relative exponent from $⟨ 2 ⟩$ to $t$ and the monomial relative exponent from $t$ to $⟨ 2 ⟩$ ,

(17)

iM(t)\coloneqqω(⟨2⟩,t)ωM(t,⟨2⟩).

Equivalently, we have

(18)

(This notion may depend on the tensor and not only on the support.)

Proposition 7.

Let $s$ , $t$ and $u$ be tensors.

[label=()]
$ω_{M} (t, t) = 1$ .
$ω_{M} (s, t) ω_{M} (t, u) \geq ω_{M} (s, u)$ (triangle inequality).
$ω_{M} (s, t) \geq ω (s, t)$ .
$i_{M} (t) \geq i (t)$ .

Definition 8.

Let $t$ be a tensor.

If $i_{M} (t) = 1$ , then we say that $t$ is monomially reversible.
If $i_{M} (t) > 1$ , then we say that $t$ is monomially irreversible.

There exist tensors that are reversible and monomially irreversible. For example, let $C$ be the structure tensor of the algebra $C [Z / 3 Z]$ in the natural basis,

(19)

C = e_{0, 0, 0} + e_{0, 1, 1} + e_{1, 0, 1} + e_{2, 0, 2} + e_{0, 2, 2} + e_{1, 1, 2} + e_{1, 2, 0} + e_{2, 1, 0} + e_{2, 2, 1} .

Then we have $\lx@converttounder\wtildeR(C)=3$ , $\lx@converttounder\wtildeQ(C)=3$ and $\lx@converttounder\wtildeQM(C)=2.75..$ (this is proven in [EG17, Tao16], see also [CVZ18] for the connection to [Str91]), so that $i (C) = 1$ and $i_{M} (C) = 1.08..$

With regards to matrix multiplication, the standard construction for (13) in fact shows that

(20)

ω_{M} (⟨ 2, 2, 2 ⟩, ⟨ 2 ⟩) = \frac{1}{2} .

2.5. Balanced tensors

We finish this section with a general comment on upper bounds on irreversibility. A tensor $t \in V_{1} \otimes V_{2} \otimes V_{3}$ with $dim (V_{1}) = dim (V_{2}) = dim (V_{3})$ is called balanced if the corresponding maps $t_{1} : V_{1} \to V_{2} \otimes V_{3}$ , $t_{2} : V_{2} \to V_{1} \otimes V_{3}$ and $t_{3} : V_{3} \to V_{1} \otimes V_{2}$ (called flattenings) are full-rank and for each $i \in [3]$ there is an element $v \in V_{i}$ such that $t_{i} (v)$ has full-rank [Str88, page 121]. For any tensor space with cubic format $F^{n} \otimes F^{n} \otimes F^{n}$ over an algebraically closed field $F$ , being balanced is a generic condition, i.e. almost all elements in such a space are balanced. Balanced tensors are called 1-generic tensors in [LM17]. Let $t \in F^{n} \otimes F^{n} \otimes F^{n}$ be balanced. Then [Str88, Proposition 3.6]

(21)		$\lx@converttounder\wtildeR(t)≤n23ω$
(22)		$\lx@converttounder\wtildeQ(t)≥n23$

and so

(23)

i (t) \leq ω .

If moreover $\lx@converttounder\wtildeR(t)=n$ , then

(24)

i (t) \leq 3 / 2.

3. Barriers through irreversibility

With the new notion of irreversibility available, we present a barrier for approaches to upper bound $ω$ via an intermediate tensor $t$ .

3.1. The irreversibility barrier

For any tensor $t$ the inequality

(25)

ω (⟨ 2 ⟩, t) ω (t, ⟨ 2, 2, 2 ⟩) \geq ω

holds by the triangle inequality. Any such approach to upper bound $ω$ respects the following barrier in terms of the irreversibility $i (t)$ of $t$ .

Theorem 9.

For any tensor $t$ holds

(26)

ω (⟨ 2 ⟩, t) ω (t, ⟨ 2, 2, 2 ⟩) \geq 2 i (t) .

Proof.

By the triangle inequality (Proposition 3),

(27)

ω (⟨ 2 ⟩, t) ω (t, ⟨ 2, 2, 2 ⟩) ω (⟨ 2, 2, 2 ⟩, ⟨ 2 ⟩) \geq ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩) = i (t) .

Therefore, using the fact $ω (⟨ 2, 2, 2 ⟩, ⟨ 2 ⟩) = \frac{1}{2}$ from (13), we have

(28)

ω (⟨ 2 ⟩, t) ω (t, ⟨ 2, 2, 2 ⟩) \geq \frac{i (t)}{ω (⟨ 2, 2, 2 ⟩, ⟨ 2 ⟩)} = 2 i (t) .

This proves the claim. ∎

Theorem 9, in particular, implies that if $i (t) > 1$ , then $ω (⟨ 2 ⟩, t) ω (t, ⟨ 2, 2, 2 ⟩) > 2$ , i.e. one cannot prove $ω = 2$ via any fixed irreversible intermediate tensor. (Of course one can consider sequences of intermediate tensors with irreversibility converging to 1.)

3.2. Better barriers through more structure

Naturally, we should expect that imposing more structure on the approach to upper bound $ω$ leads to stronger barriers. In this section we impose that the final step of the approach is an application of the Schönhage $τ$ -theorem. The Schönhage $τ$ -theorem implies that for any $α, β \in N$ and any tensor $t$ holds that

(29)

\frac{ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{β}) - α}{β} \geq ω .

We prove the following barrier in terms of $α, β$ and the irreversibility $i (t)$ of $t$ .

Theorem 10.

For any tensor $t$ and $α, β \in N$ holds

(30)

Proof.

By the triangle inequality,

(31)

ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{β}) ω (⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{β}, ⟨ 2 ⟩) \geq ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩) = i (t) .

Therefore,

(32)

ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{β}) \geq \frac{i (t)}{ω (⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{β}, ⟨ 2 ⟩)} = (α + 2 β) i (t) .

Subtracting $α$ , dividing by $β$ and using that $i (t) - 1 \geq 0$ (Proposition 5) gives the barrier

(33)

\frac{ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{β}) - α}{β} \geq \frac{(α + 2 β) i (t) - α}{β} \geq 2 i (t) + \frac{α}{β} (i (t) - 1) .

This proves the claim. ∎

As a corollary of the above theorem we present a barrier on any approach of the following form. The Schönhage $τ$ -theorem implies that for any $a, b, c \in N_{\geq 1}$ and any tensor $t$ holds

(34)

\frac{ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩^{α} ⟨ a, b, c ⟩) - α}{\frac{1}{3} {log}_{2} (a b c)} \geq ω .

We prove the following barrier in terms of $a, b, c$ , $α$ and the irreversibility of the cyclically symmetrized $cyc(t)\coloneqqt⊗((1,2,3)⋅t)⊗((1,2,3)2⋅t)$ .

Corollary 11.

For any tensor $t$ and $α \in N$ and $a, b, c \in N_{\geq 1}$ holds

(35)		$\frac{ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩^{α} ⟨ a, b, c ⟩) - α}{\frac{1}{3} {log}_{2} (a b c)}$	$\geq 2 i (cyc (t)) + \frac{α}{\frac{1}{3} {log}_{2} (a b c)} (i (c y c (t)) - 1)$
(36)			$\geq 2 i (cyc (t)) .$

One verifies that $i (t) \geq i (cyc (t))$ . If $t$ is cyclically symmetric, then $cyc (t) = t^{\otimes 3}$ and we have the equality $i (t) = i (cyc (t))$ .

Proof.

One verifies directly that $ω (⟨ 2 ⟩, t) \geq ω (⟨ 2 ⟩, cyc (t)^{\frac{1}{3}})$ and

(37)

ω (t, ⟨ 2 ⟩^{α} ⟨ a, b, c ⟩) \geq ω (cyc (t)^{\frac{1}{3}}, ⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{\frac{1}{3} {log}_{2} (a b c)}) .

Note that we are using rational powers here, which is justified by taking large enough powers of the relevant tensors. Using both inequalities and then applying Theorem 10 gives

(38)		$\frac{ω (⟨ 2 ⟩, t) ω (t, ⟨ 2 ⟩^{α} ⟨ a, b, c ⟩) - α}{\frac{1}{3} {log}_{2} (a b c)}$	$\geq \frac{ω (⟨ 2 ⟩, c y c (t)) ω (cyc (t), ⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{\frac{1}{3} {log}_{2} (a b c)}) - α}{\frac{1}{3} {log}_{2} (a b c)}$
(39)			$\geq 2 i (cyc (t)) .$

This proves the statement of the theorem. ∎

Remark 12.

For cyclically symmetric tensors $t$ our Corollary 11 implies the lower bound

(40)

ω_{g} (t) \geq ω_{u} (t) \geq 2 i (t),

on the parameter $ω_{g}$ (and the “universal” version $ω_{u}$ ) studied in [AW18b], which is a significant improvement over the barrier

(41)

ω_{g} (t) \geq \frac{3}{\frac{1}{2 i (t)} + 1}

proven in [AW18b, Theorem IV.1].

3.3. Better barriers through monomial irreversibility

Finally, we impose as an extra constraint that the transformation from the intermediate tensor $t$ to the matrix multiplication tensor happens via monomial restriction (Section 2.4), i.e. we consider the approach

(42)

ω (⟨ 2 ⟩, t) ω_{M} (t, ⟨ 2, 2, 2 ⟩) \geq ω

and the more structured approaches

(43)

\frac{ω (⟨ 2 ⟩, t) ω_{M} (t, ⟨ 2 ⟩^{α} ⟨ 2, 2, 2 ⟩^{β}) - α}{β} \geq ω

and

(44)

\frac{ω (⟨ 2 ⟩, t) ω_{M} (t, ⟨ 2 ⟩^{α} ⟨ a, b, c ⟩) - α}{\frac{1}{3} {log}_{2} (a b c)} \geq ω .

The proofs in the previous sections can be directly adapted to prove:

Theorem 13.

For any tensor $t$ holds

(45)

ω (⟨ 2 ⟩, t) ω_{M} (t, ⟨ 2, 2, 2 ⟩) \geq 2 i_{M} (t) .

Theorem 14.

For any tensor $t$ and $α, β \in N$ holds

(46)

Corollary 15.

For any tensor $t$ and $α \in N$ and $a, b, c \in N_{\geq 1}$ holds

(47)		$\frac{ω (⟨ 2 ⟩, t) ω_{M} (t, ⟨ 2 ⟩^{α} ⟨ a, b, c ⟩) - α}{\frac{1}{3} {log}_{2} (a b c)}$	$\geq 2 i_{M} (cyc (t)) + \frac{α}{\frac{1}{3} {log}_{2} (a b c)} (i_{M} (cyc (t)) - 1)$
(48)			$\geq 2 i_{M} (cyc (t)) .$

4. Explicit irreversibility lower bounds

We have seen how barriers arise from lower bounds on irreversibility. In this section we compute lower bounds on the irreversibility of two well-known intermediate tensors that play a crucial role in the best upper bounds on $ω$ : the small and big Coppersmith–Winograd tensors.

4.1. Irreversibility and the asymptotic spectrum of tensors

We begin with a general discussion of how to compute irreversibility. The asymptotic spectrum of tensors is the set of $\leq$ -monotone semiring homomorphisms from the semiring of tensors (with tensor product and direct sum as multiplication and addition) to the nonnegative reals,

(49)

Δ = {F \in Hom ({tensors}, R_{\geq 0}) : a \leq b \Rightarrow F (a) \leq F (b)} .

Strassen proves in [Str88] that $\lx@converttounder\wtildeQ(t)=minF∈ΔF(t)$ and $\lx@converttounder\wtildeR(t)=maxF∈ΔF(t)$ and he also proves (implicitly) that $ω (s, t) = {max}_{F \in Δ} {log}_{2} F (t) / {log}_{2} F (s)$ . From this we directly obtain:

Proposition 16.

Let $t$ be a tensor. Then

(50)

i (t) = \frac{{max}_{F \in Δ} log F (t)}{{min}_{F \in Δ} log F (t)} .

In an ideal world we would know $Δ$ and use it to compute $i (t)$ (or better, we would use it to compute $ω$ ). In practice we currently only have partial knowledge of $Δ$ . This partial knowledge is easiest to describe in terms of the best known lower bounds on $\lx@converttounder\wtildeR(t)$ and the best known upper bounds on $\lx@converttounder\wtildeQ(t)$ . The best known lower bounds on $\lx@converttounder\wtildeR(t)$ are simply the matrix ranks of each of the three flattenings $t_{1}, t_{2}, t_{3}$ of $t$ as described in Section 2.5. For arbitrary fields, the best general upper bounds on $\lx@converttounder\wtildeQ(t)$ that we are aware of are the Strassen upper support functionals $ζ^{θ}$ from [Str91], which we will define and use in the next section. They relate asymptotically to slice rank via [CVZ18]

(51)

\lx@converttounder\wtildeQ(t)≤limsupnslicerank(t⊗n)1/n≤minθζθ(t).

We are not aware of any example for which any of the inequalities in (51) is strict. For oblique tensors the right inequality is an equality [CVZ18] and for tight tensors both inequalities are equalities [Str91]. We thus have:

Proposition 17.

Let $t$ be a tensor. Then

(52)

i (t) \geq \frac{{max}_{i} {log}_{2} R (t_{i})}{} min θ {log}_{2} ζ^{θ} (t) .

For complex tensors we have a deeper understanding of the theory of upper bounds on the asymptotic subrank, via the quantum functionals $F^{θ}$ introduced in [CVZ18]. The quantum functionals satisfy $F^{θ} \leq ζ^{θ}$ and their minimum equals the asymptotic slice rank [CVZ18], i.e.

(53)

\lx@converttounder\wtildeQ(t)≤limsupnslicerank(t⊗n)1/n=minθFθ(t)≤minθζθ(t).

For free tensors the right inequality in (53) is an equality [CVZ18]. We thus have:

Proposition 18.

If $t$ is complex, then

(54)

i (t) \geq \frac{{max}_{i} {log}_{2} R (t_{i})}{} min θ {log}_{2} F^{θ} (t) .

4.2. Irreversibility of Coppersmith–Winograd tensors

We now compute lower bounds for the irreversibility of the Coppersmith–Winograd tensors. As mentioned, we will use the support functionals of Strassen [Str91] in our computation to upper bound the asymptotic subrank. For any $θ \in R_{\geq 0}^{3}$ with $θ_{1} + θ_{2} + θ_{3} = 1$ the upper support functional $ζ^{θ}$ is defined as

(55)		$ζθ(t)\coloneqq2ρθ(t)$
(56)		$ρθ(t)\coloneqqmins≅tmaxP∈P(supp(s))3∑i=1θiH(Pi),$

where the minimum is over all tensors $s$ isomorphic to $t$

, the maximum is over all probability distributions on the support of

s

in the standard basis, and

H (P_{i})

denotes the Shannon entropy of the

i

th marginal of

P

. Strassen proves in [Str91] that

1/ω(t,⟨2⟩)=log2\lx@converttounder\wtildeQ(t)≤ρθ(t)

(Besides from the Strassen support functionals, upper bound on the asymptotic subrank of complex tensors may be obtained from the quantum functionals. For the tensors in Theorem 19 and Theorem 21, however, this will give the same bound, since these tensors are free [CVZ18, Section 4.3].)

Theorem 19 (Small Coppersmith–Winograd tensors [Cw90, Section 6]).

For the small Coppersmith–Winograd tensor

(57)

cwq\coloneqqq∑i=1e0,i,i+ei,0,i+ei,i,0

the lower bound

(58)

2 i ({c w}_{q}) \geq \frac{2 {log}_{2} (q + 1)}{{log}_{2} 3 - \frac{2}{3} + \frac{2}{3} {log}_{2} q}

holds.

Proof.

The rank of each flattening of ${c w}_{q}$ equals $q + 1$ . Therefore, $\lx@converttounder\wtildeR(cwq)≥q+1$ . To upper bound the asymptotic subrank $\lx@converttounder\wtildeQ(cwq)$ one can upper bound the Strassen upper support functional with $θ = (1 / 3, 1 / 3, 1 / 3)$ as in [CVZ18, Example 4.22] by

(59)

ρ^{θ} ({c w}_{q}) \leq {log}_{2} 3 - \frac{2}{3} + \frac{2}{3} {log}_{2} q .

We find that

(60)

2 i ({c w}_{q}) \geq \frac{2 {log}_{2} (q + 1)}{{log}_{2} 3 - \frac{2}{3} + \frac{2}{3} {log}_{2} q} .

This proves the theorem. ∎

Remark 20.

If $q > 2$ , then the right-hand side of (58) is at least $2.02..$ See the table in Section 1 for more values. If $q = 2$ , however, then the right-hand side of (58) equals 2. Theorem 19 thus does not rule out using ${c w}_{2}$ to prove that $ω = 2$ . Indeed, as observed in [CW90, Section 11]), if $ω (⟨ 2 ⟩, {c w}_{2}) = {log}_{2} 3$ , then $ω = 2$ .

Currently, the best upper bound we have on $ω (⟨ 2 ⟩, {c w}_{q})$ is ${log}_{2} (q + 2)$ . If $ω (⟨ 2 ⟩, {c w}_{q}) = {log}_{2} (q + 2)$ , then instead of (58) we get the better barrier

(61)

2 i ({c w}_{q}) \geq \frac{2 {log}_{2} (q + 2)}{{log}_{2} 3 - \frac{2}{3} + \frac{2}{3} {log}_{2} q} .

The right-hand side of (61) has a minimum value of

(62)

\frac{18}{5 {log}_{2} 3} = 2.27..

attained at $q = 6$ .

Theorem 21 (Big Coppersmith–Winograd tensors [Cw90, Section 7]).

For the big Coppersmith–Winograd tensor

(63)

CWq\coloneqqe0,0,q+1+e0,q+1,0+eq+1,0,0+q∑i=1e0,i,i+ei,0,i+ei,i,0

the lower bound

(64)

holds, where

(65)

f(x)\coloneqq−(23−qx)log2(23−qx)−q2xlog2(2x)−(13−qx)log2(13−qx).

Proof.

The rank of each flattening of ${C W}_{q}$ equals $q + 2$ , which coincides with the well-known border rank upper bound $R - - ({C W}_{q}) \leq q + 2$ . Therefore, $\lx@converttounder\wtildeR(CWq)=q+2$ .

To upper bound the asymptotic subrank $\lx@converttounder\wtildeQ(CWq)$ we use the Strassen upper support functional with $θ = (1 / 3, 1 / 3, 1 / 3)$ . In the standard basis, the support of ${C W}_{q}$ is the set

(66)

{(0, i, i), (i, 0, i), (i, i, 0) : i \in [q]} \cup {(0, 0, q + 1), (0, q + 1, 0), (q + 1, 0, 0)} .

The symmetry implies that we can assign probability $x$ to each of $(0, i, i)$ , $(i, 0, i)$ and $(0, i, i)$ , and $\frac{1}{3} - q x$ to $(0, 0, q + 1)$ , $(0, q + 1, 0)$ and $(q + 1, 0, 0)$ . This leads to an average marginal entropy of $f (x)$ as defined in the theorem statement. The maximum of $f (x)$ is attained at

(67)

This proves the theorem. ∎

Remark 22.

The lowest value of the right-hand side of (64) is $2.16..$ attained at $q = 1$ . See the table in Section 1 for more values.

Remark 23.

A lower bound on the irreversibility of the tensors $t_{n}$ mentioned in the introduction follows directly from the results in [Str91, Theorem 6.7].

4.3. Monomial irreversibility of structure tensors of finite group algebras

We now discuss irreversibility and monomial irreversibility in the context of the group-theoretic approach developed in [CU03]. This approach produces upper bounds on $ω$ via intermediate tensors that are structure tensors of complex group algebras of finite groups. Let $⟨ G ⟩$ denote the structure tensor of the complex group algebra $C [G]$ of the finite group $G$ , in the standard basis. The group-theoretic approach (in particular [CU03, Theorem 4.1]) produces an inequality of the form

(68)

⟨ G ⟩ \geq_{M} ⟨ a, b, c ⟩

which ultimately leads to the bound

(69)

\frac{ω (⟨ 2 ⟩, ⟨ G ⟩) ω_{M} (⟨ G ⟩, ⟨ a, b, c ⟩)}{\frac{1}{3} {log}_{2} (a b c)} \geq ω

where $\geq_{M}$ and $ω_{M}$ are the monomial restriction and monomial relative exponent defined in Section 2.4.

Now the monomial irreversibility barrier from Section 3.3 comes into play. Upper bounds on the monomial asymptotic subrank of $⟨ G ⟩$ have (using different terminology) been obtained in [BCC $^{+}$ 17a, BCC $^{+}$ 17b, Saw17]. Those upper bounds imply that $⟨ G ⟩$ is monomially irreversible for every nontrivial finite group $G$ . Together with our results in Section 3.3 and the fact that the tensor $⟨ G ⟩$ is symmetric up to a permutation of the basis of one of the tensor legs, this directly leads to nontrivial barriers for the left-hand side of (69) for any fixed nontrivial group $G$ , thus putting the work of [BCC $^{+}$ 17a, BCC $^{+}$ 17b, Saw17] in a broader context. We have not tried to numerically optimise the monomial irreversibility barriers for group algebras.

Finally we mention that the irreversibility barrier (rather than the monomial irreversibility barrier) does not rule out obtaining $ω = 2$ via $⟨ G ⟩$ . Namely, $⟨ G ⟩$ is isomorphic to a direct sum of matrix multiplication tensors, $⟨ G ⟩ ≅ ⨁_{i} ⟨ d_{i}, d_{i}, d_{i} ⟩$ and, therefore, $i (⟨ G ⟩) = ({log}_{2} \sum_{i} d_{i}^{ω}) / ({log}_{2} \sum_{i} d_{i}^{2})$ . Thus, if $ω = 2$ , then $⟨ G ⟩$ is reversible,

Acknowledgements

MC acknowledges financial support from the European Research Council (ERC Grant Agreement No. 337603) and VILLUM FONDEN via the QMATH Centre of Excellence (Grant No. 10059). PV acknowledges support from the Hungarian National Research, Development and Innovation Office (NKFIH) grant no. K124152 and the Hungarian Academy of Sciences Lendület-Momentum grant for Quantum Information Theory, no. 96 141. This research was supported by the National Research, Development and Innovation Fund of Hungary within the Quantum Technology National Excellence Program (Project Nr. 2017-1.2.1-NKP-2017-00001). This material is based upon work supported by the National Science Foundation under Grant No. DMS-1638352.

References

[AFLG15] Andris Ambainis, Yuval Filmus, and François Le Gall. Fast matrix multiplication: limitations of the Coppersmith-Winograd method (extended abstract). In
STOC’15—Proceedings of the 2015 ACM Symposium on Theory of Computing
, pages 585–593. ACM, New York, 2015.
[AW18a] Josh Alman and Virginia Vassilevska Williams. Further Limitations of the Known Approaches for Matrix Multiplication. In Anna R. Karlin, editor, 9th Innovations in Theoretical Computer Science Conference (ITCS 2018), volume 94 of Leibniz International Proceedings in Informatics (LIPIcs), pages 25:1–25:15, Dagstuhl, Germany, 2018. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik.
[AW18b] Josh Alman and Virginia Vassilevska Williams. Limits on All Known (and Some Unknown) Approaches to Matrix Multiplication. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), pages 580–591, Oct 2018. arXiv:1810.08671.
[BCC $^{+}$ 17a] Jonah Blasiak, Thomas Church, Henry Cohn, Joshua A. Grochow, Eric Naslund, William F. Sawin, and Chris Umans. On cap sets and the group-theoretic approach to matrix multiplication. Discrete Anal., 2017. arXiv:1605.06702.
[BCC $^{+}$ 17b] Jonah Blasiak, Thomas Church, Henry Cohn, Joshua A Grochow, and Chris Umans. Which groups are amenable to proving exponent two for matrix multiplication? arXiv, 2017. arXiv:1712.02302.
[BPR $^{+}$ 00] Charles H. Bennett, Sandu Popescu, Daniel Rohrlich, John A. Smolin, and Ashish V. Thapliyal. Exact and asymptotic measures of multipartite pure-state entanglement. Phys. Rev. A, 63(1):012307, 2000.
[CU03] Henry Cohn and Christopher Umans. A group-theoretic approach to fast matrix multiplication. In Foundations of Computer Science, 2003. Proceedings. 44th Annual IEEE Symposium on, pages 438–449. IEEE, 2003.
[CU13] Henry Cohn and Christopher Umans. Fast matrix multiplication using coherent configurations. In Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1074–1086. SIAM, 2013.
[CVZ18] Matthias Christandl, Péter Vrana, and Jeroen Zuiddam. Universal points in the asymptotic spectrum of tensors. In Proceedings of 50th Annual ACM SIGACT Symposium on the Theory of Computing (STOC’18). ACM, New York, 2018. arXiv:1709.07851.
[CW90] Don Coppersmith and Shmuel Winograd. Matrix multiplication via arithmetic progressions. J. Symbolic Comput., 9(3):251–280, 1990.
[DVC00] Wolfgang Dür, Guivre Vidal, and Juan Ignacio Cirac. Three qubits can be entangled in two inequivalent ways. Phys. Rev. A (3), 62(6):062314, 12, 2000.
[EG17] Jordan S. Ellenberg and Dion Gijswijt. On large subsets of $F_{q}^{n}$ with no three-term arithmetic progression. Ann. of Math. (2), 185(1):339–343, 2017.
[HHHH09] Ryszard Horodecki, Paweł Horodecki, Michał Horodecki, and Karol Horodecki. Quantum entanglement. Rev. Modern Phys., 81(2):865–942, 2009.
[LG14] François Le Gall. Powers of tensors and fast matrix multiplication. In ISSAC 2014—Proceedings of the 39th International Symposium on Symbolic and Algebraic Computation, pages 296–303. ACM, New York, 2014.
[LM17] J.M. Landsberg and Mateusz Michałek. Abelian tensors. Journal de Mathématiques Pures et Appliquées, 108(3):333 – 371, 2017. arXiv:1504.03732.
[Saw17] Will Sawin. Bounds for matchings in nonabelian groups. arXiv, 2017. arXiv:1702.00905.
[Sto10] Andrew James Stothers. On the complexity of matrix multiplication. PhD thesis, University of Edinburgh, 2010. http://hdl.handle.net/1842/4734.
[Str87] Volker Strassen. Relative bilinear complexity and matrix multiplication. J. Reine Angew. Math., 375/376:406–443, 1987.
[Str88] Volker Strassen. The asymptotic spectrum of tensors. J. Reine Angew. Math., 384:102–152, 1988.
[Str91] Volker Strassen. Degeneration and complexity of bilinear maps: some asymptotic spectra. J. Reine Angew. Math., 413:127–180, 1991.
[Tao16] Terence Tao. A symmetric formulation of the Croot–Lev–Pach–Ellenberg–Gijswijt capset bound. https://terrytao.wordpress.com, 2016.
[VC15] Péter Vrana and Matthias Christandl. Asymptotic entanglement transformation between W and GHZ states. J. Math. Phys., 56(2):022204, 12, 2015.
[VDDMV02] F. Verstraete, J. Dehaene, B. De Moor, and H. Verschelde. Four qubits can be entangled in nine different ways. Phys. Rev. A (3), 65(5, part A):052112, 5, 2002.
[Wil12] Virginia Vassilevska Williams. Multiplying matrices faster than Coppersmith-Winograd. Extended abstract. In STOC’12—Proceedings of the 2012 ACM Symposium on Theory of Computing, pages 887–898. ACM, New York, 2012.

Barriers for fast matrix multiplication from irreversibility

Authors

Related Research

Barriers for rectangular matrix multiplication

Limits on All Known (and Some Unknown) Approaches to Matrix Multiplication

Larger Corner-Free Sets from Combinatorial Degenerations

The Computational Complexity of Plethysm Coefficients

Limits on the Universal Method for Matrix Multiplication

Tensor network complexity of multilinear maps

Communication Complexity, Corner-Free Sets and the Symmetric Subrank of Tensors

This week in AI

1. Introduction

2. Irreversibility

2.1. Standard definitions

2.2. Relative exponent

Assumption 1.

Definition 2.

Proposition 3.

2.3. Irreversibility.

Definition 4.

Proposition 5.

Definition 6.

2.4. Monomial relative exponent and monomial irreversibility

Proposition 7.

Definition 8.

2.5. Balanced tensors

3. Barriers through irreversibility

3.1. The irreversibility barrier

Theorem 9.

Proof.

3.2. Better barriers through more structure

Theorem 10.

Proof.

Corollary 11.

Proof.

Remark 12.

3.3. Better barriers through monomial irreversibility

Theorem 13.

Theorem 14.

Corollary 15.

4. Explicit irreversibility lower bounds

4.1. Irreversibility and the asymptotic spectrum of tensors

Proposition 16.

Proposition 17.

Proposition 18.

4.2. Irreversibility of Coppersmith–Winograd tensors

Theorem 19 (Small Coppersmith–Winograd tensors [Cw90, Section 6]).

Proof.

Remark 20.

Theorem 21 (Big Coppersmith–Winograd tensors [Cw90, Section 7]).

Proof.

Remark 22.

Remark 23.

4.3. Monomial irreversibility of structure tensors of finite group algebras

Acknowledgements

References