Skip to main content
Springer Nature Link
Log in

High Performance Tensor–Vector Multiplication on Shared-Memory Systems

  • Conference paper
  • First Online:
Parallel Processing and Applied Mathematics (PPAM 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12043))

  • 1085 Accesses

Abstract

Tensor–vector multiplication is one of the core components in tensor computations. We have recently investigated high performance, single core implementation of this bandwidth-bound operation. Here, we investigate its efficient, shared-memory implementations. Upon carefully analyzing the design space, we implement a number of alternatives using OpenMP and compare them experimentally. Experimental results on up to 8 socket systems show near peak performance for the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bader, B.W., Kolda, T.G.: Algorithm 862: MATLAB tensor classes for fast algorithm prototyping. ACM TOMS 32(4), 635–653 (2006)

    Article  MathSciNet  Google Scholar 

  2. Ballard, G., Knight, N., Rouse, K.: Communication lower bounds for matricized tensor times Khatri-Rao product. In: 2018 IPDPS, pp. 557–567. IEEE (2018)

    Google Scholar 

  3. Grama, A.Y., Gupta, A., Kumar, V.: Isoefficiency: measuring the scalability of parallel algorithms and architectures. IEEE Parallel Distrib. Technol. Syst. Appl. 1(3), 12–21 (1993)

    Article  Google Scholar 

  4. Kjolstad, F., Kamil, S., Chou, S., Lugato, D., Amarasinghe, S.: The tensor algebra compiler. Proc. ACM Program. Lang. 1(OOPSLA), 77:1–77:29 (2017)

    Article  Google Scholar 

  5. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    Article  MathSciNet  Google Scholar 

  6. Li, J., Battaglino, C., Perros, I., Sun, J., Vuduc, R.: An input-adaptive and in-place approach to dense tensor-times-matrix multiply. In: SC 2015, pp. 76:1–76:12 (2015)

    Google Scholar 

  7. Matthews, D.: High-performance tensor contraction without transposition. SIAM J. Sci. Comput. 40(1), C1–C24 (2018)

    Article  MathSciNet  Google Scholar 

  8. Morton, G.M.: A computer oriented geodetic data base and a new technique in file sequencing (1966)

    Google Scholar 

  9. Pawłowski, F., Uçar, B., Yzelman, A.J.N.: High performance tensor-vector multiples on shared memory systems. Technical report 9274, Inria, Grenoble-Rhône-Alpes (2019)

    Google Scholar 

  10. Pawlowski, F., Uçar, B., Yzelman, A.N.: A multi-dimensional Morton-ordered block storage for mode-oblivious tensor computations. J. Comput. Sci. (2019). https://doi.org/10.1016/j.jocs.2019.02.007

    Article  MathSciNet  Google Scholar 

  11. Solomonik, E., Matthews, D., Hammond, J.R., Stanton, J.F., Demmel, J.: A massively parallel tensor contraction framework for coupled-cluster computations. J. Parallel Distrib. Comput. 74(12), 3176–3190 (2014)

    Article  Google Scholar 

  12. Springer, P., Bientinesi, P.: Design of a high-performance GEMM-like tensor-tensor multiplication. ACM TOMS 44(3), 1–29 (2018)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Huawei Technologies France, 20 Quai du Point du Jour, 92100, Boulogne-Billancourt, France

    Filip Pawłowski & Albert-Jan Yzelman

  2. ENS Lyon, Lyon, France

    Filip Pawłowski & Bora Uçar

  3. CNRS and LIP (UMR5668, CNRS - ENS Lyon - UCB Lyon 1 - INRIA), Lyon, France

    Bora Uçar

Authors
  1. Filip Pawłowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bora Uçar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Albert-Jan Yzelman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Filip Pawłowski .

Editor information

Editors and Affiliations

  1. Czestochowa University of Technology, Czestochowa, Poland

    Roman Wyrzykowski

  2. University of Southern California, Marina del Rey, CA, USA

    Ewa Deelman

  3. University of Tennessee, Knoxville, TN, USA

    Jack Dongarra

  4. Czestochowa University of Technology, Czestochowa, Poland

    Konrad Karczewski

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pawłowski, F., Uçar, B., Yzelman, AJ. (2020). High Performance Tensor–Vector Multiplication on Shared-Memory Systems. In: Wyrzykowski, R., Deelman, E., Dongarra, J., Karczewski, K. (eds) Parallel Processing and Applied Mathematics. PPAM 2019. Lecture Notes in Computer Science(), vol 12043. Springer, Cham. https://doi.org/10.1007/978-3-030-43229-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-43229-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-43228-7

  • Online ISBN: 978-3-030-43229-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics