Abstract
Hypercube structures are heavily used by parallel algorithms that require all-to-all communication. When communicating over a heterogeneous and irregular network, the performance obtained by the hypercube structure will depend on the matching of the hypercube structure to the topology of the underlying network. In this paper, we present strategies to build topology-based hypercubes structures. These strategies do not assume any kind of topology. They take into account the communication cost between pair of nodes to provide a performance-efficient hypercube structure. These enhanced hypercube structures help improve the performance of parallel applications that require all-to-all communication in heterogeneous networks by up to ~30%
This research was supported in part by NSF cooperative agreement ACI-9619020 through computing resources provided by the National Partnership for Advanced Computational Infrastructure at the San Diego Supercomputer Center
Chapter PDF
References
Banikazemi, M., Moorthy, V., Panda, D.K.: Efficient Collective Communication on Heterogeneous Networks of Workstations. In: Proceedings of the ICPP 1998 (August 1998)
Banikazemi, M., et al.: Communication Modeling of Heterogeneous Networks of Workstations for Performance Characterization of Collective Operations. In: Proceedings of the Heterogeneous Computing Workshop (April 1999)
Bernaschi, M., Iannello, G.: Collective Communication Operations: Experimental Results vs. Theory. Concurrency: Practice and Experience 10(5), 359–386 (1998)
Bertsekas, D.P., et al.: Optimal Communication Algorithms for Hypercubes. Journal of Parallel and Distributed Computing 11, 263–275 (1991)
Chen, C., Chen, R.: Compact Embeddings of Binary Trees into Hypercubes. Information Processing Letters 54(2), 69–72 (1995)
Figueira, S.M., Mendes, C.: Dynamically Adaptive Binomial Trees for Broadcasting in Heterogeneous Networks of Workstations. In: Daydé, M., Dongarra, J., Hernández, V., Palma, J.M.L.M. (eds.) VECPAR 2004. LNCS, vol. 3402, pp. 480–495. Springer, Heidelberg (2005)
Foster, I.: Designing and Building Parallel Programs - Concepts and Tools for Parallel Software Engineering. Addison Wesley Publishing Company, Reading (1995)
Foster, I., et al.: Wide-Area Implementation of the Message Passing Interface. Parallel Computing 24(12), 1735–1749 (1998)
Foster, I., Karonis, N.: A Grid-Enabled MPI: Message Passing in Heterogeneous Distributed Computing Systems. In: Proceedings of the Supercomputing 1998 (November 1998)
Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NPCompleteness. W. H. Freeman, San Francisco (1979)
Geist, A., Beguelin, A., Dongarra, J., Manchek, R., Jiang, W., Sunderam, V.: PVM: A Users’ Guide and Tutorial for Networked Parallel Computing. MIT Press, Cambridge (1994)
Heun, V., Mayr, E.: Efficient Dynamic Embeddings of Binary Trees into Hypercubes. Technical Report TR-98-023, International Computer Science Institute, Berkeley, California
Johnsson, S.L.: Communication Efficient Basic Linear Algebra Computations on Hypercube Architectures. Journal of Parallel and Distributed Computing 4(2), 133–172 (1987)
Karonis, N., et al.: Exploiting Hierarchy in Parallel Computer Networks to Optimize Collective Operation Performance. In: Proceedings of the 14th IPDPS, May 2000, pp. 377–384 (2000)
Kielmann, T., et al.: MagPIe: MPI’s Collective Communication Operations for Clustered Wide Area Systems. In: Proceedings of the PPoPP 1999 (May 1999)
Kielmann, T., Bal, H.E., Gorlatch, S.: Bandwidth-Efficient Collective Communication for Clustered Wide Area Systems. In: Proceedings of the IPDPS 2000 (May 2000)
Leighton, F.T.: Introduction to Parallel Algorithms and Architectures. Morgan Kaufmann, San Francisco (1992)
Livingston, M., Stout, Q.: Embeddings in Hypercubes. Mathematical and Computational Modeling 11, 222–227 (1988)
Loan, C.: Computational Frameworks for the Fast Fourier Transform. SIAM (1992)
Lowekamp, B., Beguelin, A.: ECO: Efficient Collective Operations for Communication on Heterogeneous Networks. In: Proceedings of the 10th International Parallel Processing Symposium (April 1996)
Message-Passing Interface Forum, MPI: A Message-Passing Interface Standard. International Journal of Supercomputing Applications 8(3/4) (1994)
MPICH-A Portable Implementation of MPI, http://www-unix.mcs.anl.gov/mpi/mpich/
Quinn, M.: Parallel Computing - Theory and Practice. McGraw-Hill, New York (1994)
Ranka, S., Sahni, S.: Hypercube Algorithms for Image Processing and Pattern Recognition. Springer, Heidelberg (1990)
Swarztrauber, P.: Multiprocessor FFTs. Parallel Computing 5, 197–210 (1987)
Tseng, Y., et al.: Low-Congestion Embedding of Multiple Graphs in a Hypercube. In: International Conference on Parallel and Distributed Systems, pp. 378–385 (1992)
Wolski, R., Spring, N., Hayes, J.: The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing. Journal of Future Generation Computer Systems (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Figueira, S.M., Reddi, V.J. (2005). Topology-Based Hypercube Structures for Global Communication in Heterogeneous Networks. In: Cunha, J.C., Medeiros, P.D. (eds) Euro-Par 2005 Parallel Processing. Euro-Par 2005. Lecture Notes in Computer Science, vol 3648. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11549468_109
Download citation
DOI: https://doi.org/10.1007/11549468_109
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28700-1
Online ISBN: 978-3-540-31925-2
eBook Packages: Computer ScienceComputer Science (R0)