Skip to main content
Springer Nature Link
Log in

The Input/Output Complexity of Sparse Matrix Multiplication

  • Conference paper
Algorithms - ESA 2014 (ESA 2014)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8737))

Included in the following conference series:

  • 2337 Accesses

Abstract

We consider the problem of multiplying sparse matrices (over a semiring) where the number of non-zero entries is larger than main memory. In the classical paper of Hong and Kung (STOC ’81) it was shown that to compute a product of dense U ×U matrices, \(\Theta \left( U^3 / (B \sqrt{M}) \right)\) I/Os are necessary and sufficient in the I/O model with internal memory size M and memory block size B.

In this paper we generalize the upper and lower bounds of Hong and Kung to the sparse case. Our bounds depend of the number N = nnz(A)+nnz(C) of nonzero entries in A and C, as well as the number Z =nnz(AC) of nonzero entries in AC.

We show that using \(\tilde{O} \left( \tfrac{N}{B} \min\left(\sqrt{\tfrac{Z}{M}},\tfrac{N}{M}\right) \right)\) I/Os, AC can be computed with high probability. This is tight (up to polylogarithmic factors) when only semiring operations are allowed, even for dense rectangular matrices: We show a lower bound of \(\Omega \left( \tfrac{N}{B} \min\left(\sqrt{\tfrac{Z}{M}},\tfrac{N}{M}\right) \right)\) I/Os.

While our lower bound uses fairly standard techniques, the upper bound makes use of “compressed matrix multiplication” sketches, which is new in the context of I/O-efficient algorithms, and a new matrix product size estimation technique that avoids the “no cancellation” assumption.

This work is supported by the Danish National Research Foundation under the Sapere Aude program.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Aggarwal, A., Vitter, J.S.: The Input/Output complexity of sorting and related problems. Commun. ACM 31(9), 1116–1127 (1988)

    Article  MathSciNet  Google Scholar 

  2. Alon, N., Yuster, R., Zwick, U.: Color-coding. J. ACM (4), 844–856

    Google Scholar 

  3. Alon, N., Yuster, R., Zwick, U.: Finding and counting given length cycles. Algorithmica 17(3), 209–223 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  4. Amossen, R.R., Campagna, A., Pagh, R.: Better size estimation for sparse matrix products. APPROX/RANDOM 2010, 406–419 (2010)

    Google Scholar 

  5. Amossen, R.R., Pagh, R.: Faster join-projects and sparse matrix multiplications. In: ICDT 2009, pp. 121–126. ACM (2009)

    Google Scholar 

  6. Bender, M., Brodal, G., Fagerberg, R., Jacob, R., Vicari, E.: Optimal sparse matrix dense vector multiplication in the I/O-model. TCS 47(4), 934–962 (2010)

    MATH  MathSciNet  Google Scholar 

  7. Boyer, R.S., Moore, J.S.: MJRTY - A fast majority vote algorithm. Technical Report AI81-32 (February 1, 1981)

    Google Scholar 

  8. Cohen, E.: Estimating the size of the transitive closure in linear time. In: SFCS 1994, pp. 190–200. IEEE Computer Society, Washington, DC (1994)

    Google Scholar 

  9. Cohen, E.: Structure prediction and computation of sparse matrix products. Journal of Combinatorial Optimization 2(4), 307–332 (1998)

    Article  Google Scholar 

  10. Demaine, E.D.: Cache-oblivious algorithms and data structures. In: Lecture Notes from the EEF Summer School on Massive Data Sets, June 27-July 1 (2002)

    Google Scholar 

  11. Flajolet, P., Martin, G.N.: Probabilistic Counting Algorithms for Data Base Applications. Journal of Computer and System Sciences

    Google Scholar 

  12. Greiner, G., Jacob, R.: The I/O complexity of sparse matrix dense matrix multiplication. In: López-Ortiz, A. (ed.) LATIN 2010. LNCS, vol. 6034, pp. 143–156. Springer, Heidelberg (2010)

    Google Scholar 

  13. Irony, D., Toledo, S., Tiskin, A.: Communication lower bounds for distributed-memory matrix multiplication. JPDC 64(9), 1017–1026 (2004)

    MATH  Google Scholar 

  14. Jia-Wei, H., Kung, H.T.: I/O complexity: The red-blue pebble game. In: STOC 1981, pp. 326–333. ACM (1981)

    Google Scholar 

  15. Kane, D.M., Nelson, J., Woodruff, D.P.: An optimal algorithm for the distinct elements problem. In: PODS 2010, pp. 41–52. ACM (2010)

    Google Scholar 

  16. McGregor, A.: Algorithms for Signals, book draft (2013)

    Google Scholar 

  17. Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, New York (1995)

    Book  MATH  Google Scholar 

  18. Mulmuley, K., Vazirani, U., Vazirani, V.: Matching is as easy as matrix inversion. Combinatorica 7(1), 105–113 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  19. Pagh, R.: Compressed matrix multiplication. ACM Trans. Comput. Theory 5(3), 9:1–9:17 (2013)

    Google Scholar 

  20. Pagh, R., Silvestri, F.: The input/output complexity of triangle enumeration. arXiv preprint arXiv:1312.0723 (2013)

    Google Scholar 

  21. Pietracaprina, A., Pucci, G., Riondato, M., Silvestri, F., Upfal, E.: Space-round tradeoffs for mapreduce computations. In: ICS, pp. 235–244. ACM (2012)

    Google Scholar 

  22. Rabin, M.O., Vazirani, V.V.: Maximum matchings in general graphs through randomization. J. Algorithms 10(4), 557–567 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  23. Strassen, V.: Gaussian elimination is not optimal. Numerische Mathematik 13(4), 354–356 (1969)

    Article  MATH  MathSciNet  Google Scholar 

  24. van Dongen, S.: Graph Clustering by Flow Simulation. PhD thesis, University of Utrecht (2000)

    Google Scholar 

  25. Williams, R., Yu, H.: Finding orthogonal vectors in discrete structures. In: SODA 2014, ch. 135, pp. 1867–1877 (2014)

    Google Scholar 

  26. Williams, V.V.: Multiplying matrices faster than coppersmith-winograd. In: STOC 2012, pp. 887–898. ACM, New York (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IT University of Copenhagen, Denmark

    Rasmus Pagh & Morten Stöckel

Authors
  1. Rasmus Pagh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Morten Stöckel
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Massachusetts Institute of Technology, Cambridge, MA, USA

    Andreas S. Schulz

  2. Karlsruhe Institute of Technology (KIT), Kaiserstruhe 12, 76131, Karlsruhe, Germany

    Dorothea Wagner

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pagh, R., Stöckel, M. (2014). The Input/Output Complexity of Sparse Matrix Multiplication. In: Schulz, A.S., Wagner, D. (eds) Algorithms - ESA 2014. ESA 2014. Lecture Notes in Computer Science, vol 8737. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44777-2_62

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-44777-2_62

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-44776-5

  • Online ISBN: 978-3-662-44777-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation