Fast Sparse Matrix Multiplication Raphael Yuster Haifa University

Fast Sparse Matrix Multiplication Raphael Yuster Haifa University (Oranim) Uri Zwick Tel Aviv University ESA 2004 1

Matrix multiplication j i = 2

Matrix multiplication Complexity Authors 3 n - 2. 81 n Strassen (1969) n 2. 38 Coppersmith, Winograd (1990) 3

Sparse Matrix Multiplication = n - number of rows and columns m - number of non-zero elements The distribution of the non-zero elements in the matrices is arbitrary! 4

Sparse Matrix Multiplication j k k = Each element of B is multiplied by at most n elements from A. Complexity: mn 5

Matrix multiplication Complexity Authors 2. 38 n Coppersmith, Winograd (1990) mn - 0. 7 1. 2 2+o(1) m n +n here Can we combine the two? 6

Comparison mn m 0. 7 n 1. 2+n 2 n 2. 38 Complexity = n r (m=nr) 7

A closer look at the naïve algorithm = = 8

Complexity of the naïve algorithm Complexity = where 9

Best case for naïve algorithm Regular case: 10

Worst case for naïve algorithm 11

Worst case for naïve algorithm 0 0 = 12

Rectangular Matrix multiplication p n n n p = n Coppersmith (1997): Complexity ≤ n 1. 85 p 0. 54+n 2+o(1) For p ≤ n 0. 29, complexity = n 2+o(1) !!! 13

The combined algorithm B 1 A 2 B 2 Naïve sparse Fast rectangular Assume: a 1 b 1 ≥ a 2 b 2 ≥ … ≥ anbn matrix multiplication Choose: 0≤p≤n Compute: AB = A 1 B 1+ A 2 B 2 Complexity: 14

Analysis of combined algorithm Theorem: There exists a p for which Lemma: 15

Multiplying three sparse matrices A Complexity of new algorithm: B C m 0. 64 n 1. 46+n 2+o(1) n - number of rows and columns m - number of non-zero elements 16

Applications • Computing the square of a sparse graph • Finding short cycles (YZ’ 04) • Other applications? 17

Open problems • A faster, more sophisticated, algorithm for sparse matrix multiplication? • A faster algorithm for multiplying three or more sparse matrices? • An O(m 1 - n 1+ ) transitive closure algorithm? 18