Simultaneous Matrix Multiplication by Tensor Contraction Aggregation

Abstract

We present an interpretation of the fast matrix multiplication algorithms of Coppersmith and Winograd as aggregations of tensor contractions. This point of view provides a direct connection to older constructions by Pan and to more recent work on bilinear algorithms for symmetric tensor contraction. Further, we provide a simple explicit approach for exact aggregation of tensor contractions, each one yielding an independent matrix-matrix product. These constructions and border rank adaptations thereof do not come close to improving the best known bounds on $ω$ (the matrix multiplication exponent), but provide explicit moderate-size fast matrix multiplication algorithms that achieve a better exponent than Strassen's algorithm and have potential practical value. These finite constructions improve the multiplication-count bounds of the reference algorithms compared here for selected numbers and dimensions of independent square matrix products. In particular, using $\langle m,m,m\rangle$ to denote the matrix multiplication tensor for matrices of dimension $m$, we obtain constructions that give bounds such as the border rank bound $\underline R\!\left(\langle n^2,n^2,n^2\rangle^{\oplus9}\right) \le (n+1)^6+6(n+1)^5,$ and the exact rank bound $R\!\left(\langle n^3,n^3,n^3\rangle^{\oplus14}\right)\le (n+1)^9.$ We quantify the accuracy of such constructions in finite precision, and provide border-rank constructions that reduce cost with limiting retained fractions of accuracy at least $1/2$, $1/3$, or $1/4$ relative to the classical matrix-multiplication approach under the stated error model.