Abstract
We propose lightweight middleware solutions that facilitate and simplify the execution of MPI programs across multidomain clusters. The system described in this paper leverages H2O, a distributed metacomputing framework, to route MPI message passing across heterogeneous aggregates located in different administrative or network domains. MPI programs instantiate a specially written H2O pluglet; messages that are destined for remote sites are intercepted and transparently forwarded to their final destinations. The software was written and tested in a simulated environment, with a focus on clusters behind firewalls. Qualitatively it was demonstrated that the proposed technique is indeed effective in enabling communication across firewalls by MPI programs. In addition, tests showed only a small drop in performance, acceptable considering the substantial added functionality of sharing new resources across different administrative domains.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
MPI development tools and applications for the grid. In: Workshop on Grid Applications and Programming Tools, Seattle, WA (June 2003)
Aumage, O., Mercier, G.: MPICH/MadIII: a Cluster of Clusters Enabled MPI Implementation. In: Proc. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2003), Tokyo, May 2003, pp. 26–35. IEEE, Los Alamitos (2003)
Fagg, G., Bukovsky, A., Dongarra, J.: HARNESS and fault tolerant MPI. HARNESS and fault tolerant MPI 27(11), 1479–1496 (October 2001), Available at http://icl.cs.utk.edu/publications/pub-papers/2001/harness-ftmpi-pc.pdf
Foster, I., Karonis, N.: A grid-enabled MPI: Message passing in heterogeneous distributed computing systems. In: Supercomputing 1998, Orlando, FL (November 1998)
Foster, I., Kesselman, C.: Globus: A metacomputing infrastructure toolkit. The Intl Journal of Supercomputer Applications and High Performance Computing 11(2), 115–128 (1997)
Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The physiology of the grid: An open grid services architecture for distributed systems integration (January 2002), Available at http://www.globus.org/research/papers/ogsa.pdf
Gropp, W., Lusk, E.: Reproducible measurements of MPI performance characteristics. In: Margalef, T., Dongarra, J., Luque, E. (eds.) PVM/MPI 1999. LNCS, vol. 1697, p. 11. Springer, Heidelberg (1999)
H2O Home Page, http://www.mathcs.emory.edu/dcl/h2o/
Karonis, N., Toonen, B., Foster, I.: MPICH-G2: A grid-enabled implementation of the Message Passing Interface. Journal of Parallel and Distributed Computing (JPDC) 63(5), 551–563 (2003), Available at ftp://ftp.cs.niu.edu/pub/karonis/papers/JPDCG2/JPDCG2.ps.gz
Kurzyniec, D., Wrzosek, T., Drzewiecki, D., Sunderam, V.: Towards selforganizing distributed computing frameworks: The H2O approach. Parallel Processing Letters 13(2), 273–290 (2003)
Nemeth, Z., Sunderam, V.: A comparison of conventional distributed computing environments and computational grids. In: International Conference on Computational Science (ICCS), Amsterdam (April 2002), Available at http://www.mathcs.emory.edu/harness/pub/general/zsolt1.ps.gz
Tsujita, Y., Imamura, T., Takemiya, H., Yamagishi, N.: Stampi-I/O: A flexible parallel-I/O library for heterogeneous computing environment. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J., Volkert, J. (eds.) PVM/MPI 2002. LNCS, vol. 2474, p. 288. Springer, Heidelberg (2002), Available at http://link.springer.de/link/service/series/0558/bibs/2474/24740288.htm
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hwang, P., Kurzyniec, D., Sunderam, V. (2004). Heterogeneous Parallel Computing Across Multidomain Clusters. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2004. Lecture Notes in Computer Science, vol 3241. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30218-6_47
Download citation
DOI: https://doi.org/10.1007/978-3-540-30218-6_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23163-9
Online ISBN: 978-3-540-30218-6
eBook Packages: Springer Book Archive