Article

Multilevel hierarchical matrix multiplication on clusters

Authors:

Sascha Hunold,

Thomas Rauber,

Gudula RüngerAuthors Info & Claims

ICS '04: Proceedings of the 18th annual international conference on Supercomputing

Pages 136 - 145

https://doi.org/10.1145/1006209.1006230

Published: 26 June 2004 Publication History

Get Access

Abstract

Matrix-matrix multiplication is one of the core computations in many algorithms from scientific computing or numerical analysis and many efficient realizations have been invented over the years, including many parallel ones. The current trend to use clusters of PCs or SMPs for scientific computing suggests to revisit matrix-matrix multiplication and investigate efficiency and scalability of different versions on clusters. In this paper we present parallel algorithms for matrix-matrix multiplication which are built up from several algorithms in a multilevel structure. Each level is associated with a hierarchical partition of the set of available processors into disjoint subsets so that deeper levels of the algorithm employ smaller groups of processors in parallel. We perform runtime experiments on several parallel platforms and show that multilevel algorithms can lead to significant performance gains compared with state-of-the-art methods.

References

[1]

J. Bilmes, K. Asanovic, C.-W. Chin, and J. Demmel. Optimizing matrix multiply using PHiPAC: A portable, high-performance, ANSI c coding methodology. In Proc. of the International Conference on Supercomputing -- ICS'97, pages 340--347, 1997.

Digital Library

Google Scholar

[2]

C.-C. Chou, Y.-F. Deng, G. Li, and Y. Wang. Parallelizing Strassen's Method for Matrix Multiplication on Distributed-Memory MIMD Architectures. Computers and Mathematics with Applications, 30(2):49--69, 1995.

Crossref

Google Scholar

[3]

F. Desprez and F. Suter. Impact of Mixed-Parallelism on Parallel Implementations of Strassen and Winograd Matrix Multiplication Algorithms. Technical Report RR2002-24, Laboratoire de l'Informatique du Parallélisme (LIP), June 2002. Also INRIA Research Report RR-4482.

Google Scholar

[4]

B. Dumitrescu, J.-L. Roch, and D. Trystram. Fast matrix multiplications algorithms on MIMD architectures. Parallel Algorithms and Applications, 4(2):53--70, 1994.

Crossref

Google Scholar

[5]

G. Golub and C. Van Loan. Matrix Computations.The John Hopkins University Press, 1989.

Google Scholar

[6]

B. Grayson, A. Shah, and R. van de Geijn. A High Performance Parallel Strassen Implementation. Technical Report CS-TR-95-24, Department of Computer Sciences, The Unversity of Texas, 1, 1995.

Digital Library

Google Scholar

[7]

S. Hunold, T. Rauber,and G. Rünger. Hierarchical Matrix-Matrix Multiplication based on Multiprocessor Tasks. In Proc.of theInternationalConference on Computational Science -- ICCS 2004, LNCS. Springer, June 2004.

Crossref

Google Scholar

[8]

Q.LuoandJ.B.Drake.AScalableParallel Strassen's Matrix Multiplication Algorithm for Distributed-Memory Computers. In Proceedings of the 1995 ACM Symposium on Applied Computing, pages 221--226. ACM Press, 1995.

Digital Library

Google Scholar

[9]

T. Rauber and G. Rünger. Library Support for Hierarchical Multi-Processor Tasks. In Proc. of the Supercomputing 2002, Baltimore, USA, 2002.

Digital Library

Google Scholar

[10]

V. Strassen. Gaussian elimination is not optimal. Numerische Mathematik, 13:354--356, 1969.

Digital Library

Google Scholar

[11]

R. A. van de Geijn and J. Watts. SUMMA: scalable universal matrix multiplication algorithm. Concurrency: Practice and Experience, 9(4):255--274, 1997.

Crossref

Google Scholar

[12]

R. C. Whaley and J. J. Dongarra. Automatically Tuned Linear Algebra Software. Technical Report UT-CS-97-366, University of Tennessee, 1997.

Digital Library

Google Scholar

Cited By

View all

Deng ZZhou YWu DYe GChen MXiao L(2018)Utility Maximization of Cloud-Based In-Car Video Recording Over Vehicular Access NetworksIEEE Internet of Things Journal10.1109/JIOT.2018.28441695:6(5213-5226)Online publication date: Dec-2018
https://doi.org/10.1109/JIOT.2018.2844169
Kelefouras VKritikakou AMporas IKolonias V(2016)A high-performance matrix---matrix multiplication methodology for CPU and GPU architecturesThe Journal of Supercomputing10.1007/s11227-015-1613-772:3(804-844)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1007/s11227-015-1613-7
Kelefouras VKritikakou AGoutis C(2014)A Matrix---Matrix Multiplication methodology for single/multi-core architectures using SIMDThe Journal of Supercomputing10.1007/s11227-014-1098-968:3(1418-1440)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1007/s11227-014-1098-9
Show More Cited By

Index Terms

Multilevel hierarchical matrix multiplication on clusters

Recommendations

Combining building blocks for parallel multi-level matrix multiplication

This paper presents parallel algorithms for matrix-matrix multiplication which are built up from several algorithms in a multi-level structure. The upper level consists of Strassen's algorithm which is performed for a predefined number of recursions. ...
A Divide-and-Conquer Strategy and PVM Computation Environment for the Matrix Multiplication
ICA3PP '09: Proceedings of the 9th International Conference on Algorithms and Architectures for Parallel Processing

This paper deals with parallels of the fast matrix multiplication strassen's algorithm, winograd's algorithm and analyzes empirical study of the matrix multiplication under the distributed environment in which PVM (Parallel Virtual Machine) environment ...
Extra High Speed Matrix Multiplication on the Cray-2

The Cray-2 is capable of performing matrix multiplication at very high rates. Using library routines provided by Cray Research, Inc., performance rates of 300 to 425 MFLOPS can be obtained on a single processor, depending on system load. Considerably ...

Comments

Information & Contributors

Information

Published In

ICS '04: Proceedings of the 18th annual international conference on Supercomputing

June 2004

360 pages

ISBN:1581138393

DOI:10.1145/1006209

General Chair:
Paul Feautrier
LIP, ENS Lyon
,
Program Chairs:
James Goodman
University of Auckland
,
André Seznec
IRISA, INRIA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICS04

Sponsor:

ICS04: International Conference on Supercomputing 2004

June 26 - July 1, 2004

Malo, France

Acceptance Rates

Overall Acceptance Rate 629 of 2,180 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
707
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Deng ZZhou YWu DYe GChen MXiao L(2018)Utility Maximization of Cloud-Based In-Car Video Recording Over Vehicular Access NetworksIEEE Internet of Things Journal10.1109/JIOT.2018.28441695:6(5213-5226)Online publication date: Dec-2018
https://doi.org/10.1109/JIOT.2018.2844169
Kelefouras VKritikakou AMporas IKolonias V(2016)A high-performance matrix---matrix multiplication methodology for CPU and GPU architecturesThe Journal of Supercomputing10.1007/s11227-015-1613-772:3(804-844)Online publication date: 1-Mar-2016
https://dl.acm.org/doi/10.1007/s11227-015-1613-7
Kelefouras VKritikakou AGoutis C(2014)A Matrix---Matrix Multiplication methodology for single/multi-core architectures using SIMDThe Journal of Supercomputing10.1007/s11227-014-1098-968:3(1418-1440)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1007/s11227-014-1098-9
Fadhli MGani TMelinda Away Y(2012)Comparison on efficiency of computational efforts between cluster computation (MapReduce) and single host computation2012 International Conference on Cloud Computing and Social Networking (ICCCSN)10.1109/ICCCSN.2012.6215743(1-5)Online publication date: Apr-2012
https://doi.org/10.1109/ICCCSN.2012.6215743
Nasri WSteffenel LTrystram D(2009)Adaptive approaches for efficient parallel algorithms on cluster-based systemsInternational Journal of Grid and Utility Computing10.1504/IJGUC.2009.0220261:2(98-108)Online publication date: 1-Dec-2009
https://dl.acm.org/doi/10.1504/IJGUC.2009.022026
Hoffmann RHunold SKorch MRauber T(2009)Towards Scalable Parallel Numerical Algorithms and Dynamic Load Balancing StrategiesHigh Performance Computing in Science and Engineering, Garching/Munich 200710.1007/978-3-540-69182-2_40(503-516)Online publication date: 2009
https://doi.org/10.1007/978-3-540-69182-2_40
Hafeez MYounus MRehman AMohsin A(2007)Optimal solution to matrix parenthesization problem employing parallel processing approachProceedings of the 8th Conference on 8th WSEAS International Conference on Evolutionary Computing - Volume 810.5555/1347992.1347994(235-240)Online publication date: 19-Jun-2007
https://dl.acm.org/doi/10.5555/1347992.1347994
Rauber TRünger G(2007)Mixed task and data parallel executions in general linear methodsScientific Programming10.1155/2007/68319815:3(137-155)Online publication date: 1-Aug-2007
https://dl.acm.org/doi/10.1155/2007/683198
Russek PWiatr K(2007)Dedicated architecture for double precision matrix multiplication in supercomputing environment2007 IEEE Design and Diagnostics of Electronic Circuits and Systems10.1109/DDECS.2007.4295303(1-4)Online publication date: Apr-2007
https://doi.org/10.1109/DDECS.2007.4295303
Rauber TRünger G(2006)Anticipated distributed task scheduling for grid environmentsProceedings of the 20th international conference on Parallel and distributed processing10.5555/1898699.1898883(337-337)Online publication date: 25-Apr-2006
https://dl.acm.org/doi/10.5555/1898699.1898883
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Combining building blocks for parallel multi-level matrix multiplication

A Divide-and-Conquer Strategy and PVM Computation Environment for the Matrix Multiplication

Extra High Speed Matrix Multiplication on the Cray-2

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations