Abstract
Parallel computing is moving rapidly from an era of “Big Iron” to a future that will be dominated by systems built from commodity components. Users will be able to construct high-performance systems by clustering off-the-shelf processors using widely available high-speed switches. A key question is how to organize the supporting software to best leverage these commodity hardware systems. We address this question by reviewing two aspects of how to best implement data parallel programs on commodity clusters. We identify research questions that arise at the interface between the network and the user program, and we also explore problems that arise when attempting to utilize multiprocessors as the basic building block for constructing clusters.
On leave during 1995–96 at the University of Southampton.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
R. Arpaci, A. Dusseau, A. Vahdat, L. Liu, T. Anderson, and D. Patterson. The interaction of parallel and sequential workloads on a network of workstations. Technical Report CS-94–838, University of California-Berkeley, 1994.
G. Blelloch. NESL: A nested data-parallel language. Technical Report CMU-CS-93–129, Carnegie Mellon University, 1993.
S. Chappelow, P. Hatcher, and J. Mason. Optimizing data-parallel stencil computations in a portable framework. In Languages, Compilers and Run-Time Systems for Scalable Computers. Kluwer Academic Publishers, 1995.
Jurassic Pact. The Economist, March 2–8, 1996. Pages 72–73.
J. Frankel. A reference description of the C* language. Technical Report TR-253, Thinking Machines Corporation, Cambridge, MA, 1991.
A. Grimshaw, W. Wulf, J. French, A. Weaver, and P. Reynolds Jr. Legion: The next logical step toward a nationwide virtual computer. Technical Report CS-94–21, University of Virginia, 1994.
P. Keleher, S. Dwarkadas, A. L. Cox, and W. Zwaenepoel. Treadmarks: Distributed shared memory on standard workstations and operating systems. In Proceedings of the Winter 1994 USENIX Conference, pages 115–131, January 1994.
C. Koelbel, D. Loveman, R. Schreiber, and G. Steele Jr. The High Performance Fortran Handbook. The MIT Press, 1994.
S. Kumaran, R. Miller, and M. Quinn. Architecture-adaptable finite element modelling: A case study using an ocean circulation simulation. In Proceedings of Supercomputing '95, December 1995.
S. Kumaran and M. Quinn. Machine-independent parallel programming using the divide-and-conquer paradigm. In Languages, Compilers and Run-Time Systems for Scalable Computers, pages 71–84. Kluwer Academic Publishers, 1995.
S. Kumaran and M. Quinn. Architecture-adaptable parallel processing using object technology. In Proceedings of Parallel Object-Oriented Methods and Applications '96, February 1996.
S. Kumaran and M. Quinn. Towards architecture-adaptable parallel programming. Scientific Programming, 1996. To appear.
A. Lapadula and K. Herold. A retargetable C* compiler and run-time library for mesh-connected MIMD multicomputers. Technical Report 92-15, University of New Hampshire, 1992.
D. Lenoski, J. Laudon, K. Gharachorloo, W.-D. Weber, A. Gupta, J. Hennessy, M. Horowitz, and M. S. Lam. The Stanford Dash multiprocessor. Computer, 25(3):63–79, 1992.
E. Markatos and T. LeBlanc. Using processor affinity in loop scheduling on shared-memory multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 5(4):379–400, April 1994.
S. Pakin, M. Lauria, and A. Chien. High performance messaging on workstations: Illinois fast messages (FM) for Myrinet. In Proceedings of Supercomputing '95, December 1995.
M. Squillante and E. Lazowska. Using processor-cache affinity information in shared-memory multiprocessor scheduling. IEEE Transactions on Parallel and Distributed Systems, 4(2):131–143, February 1993.
L. Stanberry, editor. Data Parallel C Extensions. Technical report, American National Standards Institute, December 1994. Available via http://www.cs.unh.edu/pjh/dpce/dpce.html.
V. Sunderam. PVM: A framework for parallel distributed computing. Concurrency: Practice and Experience, 2(4):315–339, December 1990.
T. von Eicken, A. Basu, V. Buch, and W. Vogels. U-Net: A user-level network interface for parallel and distributed computing. In Proceedings of the 15th ACM Symposium on Operating Systems Principles, December 1995.
T. von Eicken, D. Culler, S. Goldstein, and K. Schauser. Active messages: A mechanism for integrated communication and computation. In International Symposium on Computer Architectures, pages 256–266, 1992.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hatcher, P.J., Russell, R.D., Kumaran, S., Quinn, M.J. (1996). Implementing data parallel programs on commodity clusters. In: Perrin, GR., Darte, A. (eds) The Data Parallel Programming Model. Lecture Notes in Computer Science, vol 1132. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61736-1_47
Download citation
DOI: https://doi.org/10.1007/3-540-61736-1_47
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61736-5
Online ISBN: 978-3-540-70646-5
eBook Packages: Springer Book Archive