Abstract
During the software crisis of the 1960s, Dijkstra’s famous thesis “goto considered harmful” paved the way for structured programming, i.e. software development with well-defined and disciplined organization of control flow. In parallel programming, a new aspect - communication - has an important impact on the structure and properties of programs. This paper shows that many current difficulties of parallel programming are caused by complicated and poorly structured communication, which is a consequence of using low-level send-recv primitives. We argue that, like goto in sequential programs, send-recv should be avoided as far as possible and replaced by collective operations in the parallel setting. We argue against some widely held opinions about the apparent superiority of individual over collective communication and present substantial theoretical and empirical evidence to the contrary. The paper overviews some recent results on formal transformation rules for collective operations that facilitate systematic, performance-oriented design of parallel programs using MPI (Message Passing Interface).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M. Aldinucci, S. Gorlatch, C. Lengauer, and S. Pelagatti. Towards parallel programming by transformation: The FAN skeleton framework. Parallel Algorithms and Applications, 16(2):87–113, 2001.
V. Bala et al. CCL: a portable and tunable collective communication library for scalable parallel computers. In Proc. 8th Int. Conf. on Parallel Processing.
M. Bernashi, G. Iannello, and M. Lauria. Experimental results about MPI collective communication operations. In High-Performance Computing and Networking, Lecture Notes in Computer Science 1593, pages 775–783, 1999.
G. Bilardi, K. Herley, A. Pietracaprina, G. Pucci, and P. Spirakis. BSP vs. LogP. In Eighth ACM Symp. on Parallel Algorithms and Architectures, pages 25–32, 1996.
R. Bird. Lectures on constructive functional programming. In M. Broy, editor, Constructive Methods in Computing Science, NATO ASI Series F: Computer and Systems Sciences. Vol. 55, pages 151–216. Springer Verlag, 1988.
C. Böhm and G. Jacopini. Flow diagrams, turing machines and languages with only two formation rules. Comm. ACM, 9:366–371, 1966.
O.-J. Dahl, E. W. Dijkstra, and C. A.R.Hoare. Structured Programming. Academic Press, 1975.
B. Di Martino, A. Mazzeo, N. Mazzocca, and U. Villano. Restructuring parallel programs by transformation of point-to-point interactions into collective communication. Available at http://www.grid.unina.it.
E. W. Dijkstra. Go To statement considered harmful. Comm. ACM, 11(3):147–148, 1968.
A. Geist et al. PVM: Parallel Virtual Machine. MIT Press, 1994.
S. Gorlatch. Towards formally-based design of message passing programs. IEEE Trans. on Software Engineering, 26(3):276–288, March 2000.
S. Gorlatch and C. Lengauer. Abstraction and performance in the design of parallel programs: overview of the SAT approach. Acta Informatica, 36(9):761–803, 2000.
S. Gorlatch, C. Wedler, and C. Lengauer. Optimization rules for programming with collective operations. In M. Atallah, editor, Proc. IPPS/SPDP’99, pages 492–499. IEEE Computer Society Press, 1999.
M. Goudreau, K. Lang, S. Rao, T. Suel, and T. Tsantilas. Towards efficiency and portablility. programming with the BSP model. In Eighth ACM Symp. on Parallel Algorithms and Architectures, pages 1–12, 1996.
M. Goudreau and S. Rao. Single-message vs. batch communication. In M. Heath, A. Ranade, and R. Schreiber, editors, Algorithms for parallel processing, pages 61–74. Springer-Verlag, 1999.
K. Hwang and Z. Xu. Scalable Parallel Computing. McGraw Hill, 1998.
T. Kielmann, H. E. Bal, and S. Gorlatch. Bandwidth-efficient collective communication for clustered wide area systems. In Parallel and Distributed Processing Symposium (IPDPS 2000), pages 492–499, 2000.
T. Kielmann, R. F. Hofman, H. E. Bal, A. Plaat, and R. A. Bhoedjang. Mag-PIe: MPI’s collective communication operations for clustered wide area systems. In Proc. ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP’99), pages 131–140, 1999.
Y. Kolosova, V. Korneev, V. Konstantinov, and N. Mirenkov. Yazik paralleljnykh algorithmov. In Vychsliteljnye Sistemy, volume 57. Nauka, 1973. In Russian.
A. Nelisse, T. Kielmann, H. E. Bal, and J. Maassen. Object-based collective communication in java. In Joint ACM JavaGrande-ISCOPE 2001 Conference, 2001.
P. Pacheco. Parallel Programming with MPI. Morgan Kaufmann Publ., 1997.
J.-Y. L. Park, H.-A. Choi, N. Nupairoj, and L. M. Ni. Construction of optimal multicast trees based on the parameterized communication model. In Proc. Int. Conference on Parallel Processing (ICPP), volume I, pages 180–187, 1996.
S. S. Vadhiyar, G. E. Fagg, and J. Dongarra. Automatically tuned collective communications. In Proc. Supercomputing 2000. Dallas, TX, November 2000.
L. Valiant. General purpose parallel architectures. In Handbook of Theoretical Computer Science, volume A, chapter 18, pages 943–971. MIT Press, 1990.
R. van deGeijn. Using PLAPACK: Parallel Linear Algebra package. Scientific and Engineering Computation Series. MIT Press, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gorlatch, S. (2001). Send-Recv Considered Harmful? Myths and Truths about Parallel Programming. In: Malyshkin, V. (eds) Parallel Computing Technologies. PaCT 2001. Lecture Notes in Computer Science, vol 2127. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44743-1_24
Download citation
DOI: https://doi.org/10.1007/3-540-44743-1_24
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42522-9
Online ISBN: 978-3-540-44743-6
eBook Packages: Springer Book Archive