On the inequivalence of bilinear algorithms for $3\times 3$ matrix multiplication

Highlights

• A heuristic for finding algorithms for 3 by 3 matrix multiplication is presented.

• Our heuristic is very simpler and faster than the previous published methods.

• Using the heuristic, many algorithms are obtained.

• Many algorithms are proved to be inequivalent to the published ones, theoretically.

Abstract

Since Laderman showed an algorithm for $3\times 3$ matrix multiplication using 23 scalar multiplications, Johnson and McLoughlin used a numerical optimization and human controlled method to give two parameterized algorithms in which the coefficients are rational numbers. The algorithms are inequivalent to Ladermanʼs one with respect to the transformation introduced by de Groote. We present a simple and fast numerical heuristic for finding valid algorithms. Then we show that many of the obtained algorithms are inequivalent to the published ones.

Introduction

For two $2\times 2$ matrices A and B, let C be the product AB. Strassen [1] showed that seven scalar multiplications are sufficient to get C. Let

$$P_t=\left(\sum _{i,j}{\alpha }_{ijt}{A}_{ij}\right)\left(\sum _{i,j}{\beta }_{ijt}{B}_{ij}\right)$$

and

$$C_{nm}=\sum _t{\gamma }_{mnt}{P}_t$$

where $t=0,1,\dots ,6$, and ${\alpha }_{ijt},{\beta }_{ijt}\in \{-1,0,1\}$.

By finding such numbers ${\alpha }_{ijt},{\beta }_{ijt},{\gamma }_{ijt}$, he proved that 7 scalar multiplications are enough to compute C. By applying the result recursively to the problem of multiplying two $n\times n$ matrices, Strassen derived an algorithm with $O(n^{{\log }_{2}7})$ arithmetic operations which is better than $O(n^3)$, the complexity of the naive method. Later, Winograd [2] proved that 7 is the minimum number of multiplications required to multiply two $2\times 2$ matrices in the context of bilinear combinations.

One way to obtain fast algorithms for multiplying two $n\times n$ matrices is to look for other small cases such as the problem of multiplying a $2\times 2$ matrix by a $2\times 3$ matrix. For this case, it was proved that 11 multiplications are required. The upper bound was obtained by combining Strassenʼs algorithm with a vector-matrix multiplication and the lower bound was obtained by Alexeyev [3]. For the case of multiplying a $2\times 3$ matrix by a $3\times 3$ matrix, the possible number of multiplications is either 14 or 15. The upper bound of 15 was derived by Hopcroft and Kerr [4] and the lower bound of 14 was proved by Brockett and Dobkin [5].

In this paper, we consider the problem of multiplicating two $3\times 3$ matrices. For this case, Laderman [6] devised an algorithm that uses 23 multiplications instead of 27. Bläser [7] proved that a lower bound is 19. The time complexity of Ladermanʼs algorithm becomes $O(n^{{\log }_{3}23})$. Although the complexity is worse than that of Strassen algorithm, the result has an important theoretical meaning which will be discussed in Section 1.2.

Currently, the best complexity for the product of two $n\times n$ matrices is $O(n^{2.3727})$ by Williams [8].

We can think of ${\alpha }_{ijt},{\beta }_{ijt},{\gamma }_{mnt}$ as matrices. That is, ${\alpha }_{ijt}$ can be considered as a $3\times 3$ matrix where i (j resp.) denotes row index (column index, resp.) for $i,j=0,1,2$. Then, an algorithm for the multiplication of two $3\times 3$ matrices using 23 scalar multiplications consists of $23\cdot 3$ matrices. Let ${\alpha }_{ijt}^{\prime },{\beta }_{ijt}^{\prime },{\gamma }_{mnt}^{\prime }$ be another algorithm. Two algorithms, consisting of ${\alpha }_{ijt},{\beta }_{ijt},{\gamma }_{mnt}$ and ${\alpha }_{ijt}^{\prime },{\beta }_{ijt}^{\prime },{\gamma }_{mnt}^{\prime }$, respectively, are called equivalent when one can be transformed to the other using a series of transformations by permutation, cyclic permutation, transposition, scalar multiplication, and matrix multiplication [9].

For the multiplication of $2\times 2$ matrices, de Groote [10] proved that Strassen algorithm is unique under this transformation by showing that any algorithm is transformable to Strassenʼs. In other words, every algorithms of 7 scalar multiplications are equivalent to Strassenʼs.

For the $3\times 3$ matrix multiplication, Johnson and McLoughlin [11] presented two parameterized algorithms and proved that they are inequivalent to Ladermanʼs. However, their result does not give an answer to a critical question: How many inequivalent algorithms exist. In this paper, we show that there exist many algorithms inequivalent to Ladermanʼs and Johnson–McLoughlinʼs algorithms.

De Grooteʼs transformation has an invariance property for ranks of matrices, which is useful when proving inequivalence between two algorithms. Simply speaking, if the distribution of ranks for an algorithm is different from that for another algorithm, then they are inequivalent. (We just implicitly mention the whole distribution of ranks of 69 matrices. De Grooteʼs transformation has a stronger invariance property than we mentioned here.)

The Ladermanʼs algorithm consists of 51 matrices of rank 1, 12 matrices of rank 2, and 6 matrices of rank 3. For convenience, we simply write the distribution as $[51:12:6]$. In case of Johnson and McLoughlinʼs first parameterized algorithm, the distribution can be either $[47:21:1]$ or $[48:20:1]$ depending on the given parameters. The distribution of their second algorithm can be $[48:21:0]$, $[49:20:0]$ or $[50:19:0]$. From the fact that the distributions of ranks for the two Johnson and McLoughlinʼs algorithms are different to the Ladermanʼs, the algorithms are inequivalent to the Ladermanʼs. (Johnson and McLoughlinʼs first algorithm is also inequivalent to their second algorithm.) Johnson and McLoughlin used this invariance property to prove their result.

To prove inequivalence, we need to find two candidate algorithms. However it is difficult to find even one candidate algorithm. To find one algorithm, Laderman used a deeper analysis for the problem. Johnson and McLoughlin used the help of computer and a fairly delicate procedure considering de Grooteʼs transformation so that they could get two parameterized algorithms.

Since our first step is to find many candidate algorithms, the published methods mentioned above are not satisfactory in that they take too much time. We added a simple rounding method to Brentʼs numerical method so that we could get a simple and fast heuristic to find many algorithms in a short time.

Since we could find algorithms with 23 multiplications, we also tried to find an algorithm with 22 multiplications. Unfortunately, given ${\alpha }_{ijt}$ʼs, ${\beta }_{ijt}$ʼs, ${\gamma }_{mnt}$ʼs, at most 8 out of 9 $C_{nm}$ʼs could be obtained.

We first present the basic numerical method commonly used for solving (1), (2). Then we show our heuristic which adds a simple rounding method to the basic numerical method. Finally, we give just three algorithms that are inequivalent to the published ones; due to space limit, we could not list all the inequivalent algorithms that we found. Thus, before showing the three algorithms, we give a summary of the rank distributions of them.