Skip to main content
SpringerLink
Log in

A middleware approach for pipelining communications in clusters

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The Pipelining Communications Middleware (PCM) approach provides a flexible, simple, high-performance mechanism to connect parallel programs running on high performance computers or clusters. This approach enables parallel programs to communicate and coordinate with each other to address larger problems than a single program can solve. The motivation behind the PCM approach grew out of using files as an intermediate transfer stage between processing by different programs. Our approach supersedes this practice by using streaming data set transfers as an “online” communication channel between simultaneously active parallel programs. Thus, the PCM approach addresses the issue of sending data from a parallel program to another parallel program without exposing details such as number of nodes allocated to the program, specific node identifiers, etc. This paper outlines and analyzes our proposed computation and communication model to provide efficient and convenient communications between parallel programs running on high performance computing systems or clusters. We also discuss the PCM challenges as well as current PCM implementations. Our approach achieves scalability, transparency, coordination, synchronization and flow control, and efficient programming. We experimented with data parallel applications to evaluate the performance of the PCM approach. Our experiment results show that the PCM approach achieves nearly ideal throughput that scales linearly with the underlying network medium speed. PCM performs well with small and large data transfers. Furthermore, our experiments show that network infrastructure plays the most significant role in the PCM performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Beckman, P.H., Fasel, P.K., Humphrey, W.F., Mniszewski, S.M.: Efficient coupling of parallel applications using PAWS. In: The 7th IEEE International Symposium on High Performance Distributed Computing, July 1998

  2. Berman, F., Wolski, R., Casanova, H., Cirne, W., Dail, H., Faerman, M., Figueira, S., Hayes, J., Obertelli, G., Schopf, J., Shao, G., Smallen, S., Spring, N., Su, A., Zagorodnov, D.: MPICH-G2: a grid-enabled implementation of the message passing interface. IEEE Trans. Parallel Distributed Syst. 14(4), 369–382 (2003)

    Article  Google Scholar 

  3. CUMULVS, http://www.csm.ornl.gov/cs/cumulvs.html

  4. Darema-Rogers, F., George, D.A., Norton, V.A., Pfister, G.F.: Single-program-multiple-data computational model for EPEX/ FORTRAN. Technical Report RC 11552, IBM T.J. Watson Research Center, November 1986

  5. Edjlali, G., Sussman, A., Saltz, J.: Interoperability of data parallel runtime libraries. In: Proceedings of the 11th International Parallel Processing Symposium, April 1997

  6. Fide, S., Jenks, S.: Cluster-to-cluster communications: a consistent communication paradigm for distributed clusters. In: Proceedings of the 16th IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS 2004), Cambridge, MA, November 2004

  7. National Partnership for Advanced Computational Infrastructure (NPACI): NPACI ROCKS Cluster Distribution: User’s Guide, 2002, http://rocks.npaci.edu

  8. MPI Forum: Message Passing Interface (MPI)-2.0 Standard Document, http://www.mpi-forum.org

  9. Foster, I.: What is the grid? A Three Point Checklist. GRIDToday, July 2002

  10. Foster, I., Karonis, N.: A grid-enabled MPI: message passing in heterogeneous distributed computing systems. In: Proceedings 1998 Supercomputing Conference, 1998

  11. Foster, I., Karonis, N., Kesselman, C., Koenig, G., Tuecke, S.: A secure communications infrastructure for high performance distributed computing. In: Proceedings 6th IEEE Symposium on High Performance Distributed Computing, 1997

  12. Foster, I., Kesselman, C.: Globus: a metacomputing infrastructure toolkit. Int. J. Supercomp. Appl. 11(2), 115–128 (1997)

    Article  Google Scholar 

  13. Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Kaufmann, San Francisco (1998)

    Google Scholar 

  14. Geist, G.A., Kohl, J.A., Papadopoulos, P.M.: CUMULVS: providing fault-tolerance, visualization and steering of parallel applications. In: Proceedings of the Environment and Tools for Parallel Scientific Computing Workshop, August 1996

  15. Gropp, W., Huss-Lederman, S., Lumsdaine, A., Lusk, E., Nitzberg, B., Saphir, W., Snir, M.: The MPI Extensions. MPI—The Complete Reference, vol. 2. MIT Press, Cambridge (1998)

    Google Scholar 

  16. Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message Passing Interface, 2nd edn. MIT Press, Cambridge (1999)

    Book  Google Scholar 

  17. Gropp, W., Lusk, E., Thakur, R.: Using MPI-2: Advanced Features of the Message Passing Interface. MIT Press, Cambridge (1999)

    Book  Google Scholar 

  18. Myricom Inc.: Myricom Home Page, http://www.myri.com

  19. Karonis, N., Toonen, B., Foster, I.: MPICH-G2: a grid-enabled implementation of the message passing interface. J. Parallel Distributed Comput. 63(5), 551–563 (2003)

    Article  MATH  Google Scholar 

  20. Keahey, K., Fasel, P.K., Mniszewski, S.M.: PAWS: collective interactions and data transfers. In: The 10th IEEE International Symposium on High Performance Distributed Computing, August 2001

  21. Klasky, S., Ethier, S., Lin, Z., Martins, K., McCune, D., Samtaney, R.: Grid-based parallel data streaming implemented for the gyrokinetic toroidal code. In: Proceedings SC2003: High Performance Networking and Computing. IEEE Computer Society Press, Washington (2003)

    Google Scholar 

  22. Argonne National Laboratory: MPICH: A Portable MPI Implementation, http://www-unix.mcs.anl.gov/mpi/mpich/

  23. LAM/MPI Parallel Computing, http://www.lam-mpi.org

  24. Muller, M., Hess, M., Gabriel, E.: Grid enabled MPI solutions for clusters. In: Proceedings 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID’03), Tokyo, Japan, May 2003. IEEE Computer Society Press, Washington (2003)

    Google Scholar 

  25. Papadopoulos, P.M., Kohl, J.A., Semeraro, B.D.: CUMULVS: extending a generic steering and visualization middleware for application fault-tolerance. In: Proceedings of the 31st Annual Hawaii International Conference on System Sciences, January 1998

  26. PAWS, http://acts.nersc.gov/paws/

  27. Petrini, F., Feng, W.C., Hoisie, A., Coll, S., Frachtenberg, E.: The quadrics network: high-performance clustering technology. IEEE Micro, 46–57 (2002)

  28. Plesums, C.: Introduction to Workflow, 2003, http://www.e-workflow.org/bookstore/introduction_to_workflow02.pdf

  29. The Globus Project, http://www.globus.org

  30. Ranganathan, M., Acharya, A., Edjlali, G., Sussman, A., Saltz, J.: Runtime coupling of data-parallel programs. In: Proceedings of the 1996 International Conference on Supercomputing, May 1996

  31. Scalable Coherent Interface, August 1993. IEEE Standard 1596-1992

  32. Smarr, L.L., Chien, A.A., DeFanti, T., Leigh, J., Papadopoulos, P.M.: The OptIPuter. Commun. ACM 46(11), 58–67 (2003)

    Article  Google Scholar 

  33. Snir, M., Otto, S., Huss-Lederman, S., Walker, D., Dongarra, J.: The MPI Core, 2nd edn. MPI—The Complete Reference, vol. 1. MIT Press, Cambridge (1998)

    Google Scholar 

  34. Taflove, A.: Computational Electrodynamics: The Finite-Difference Time-Domain Method. Artech House, Norwood (1995)

    MATH  Google Scholar 

  35. Finite-Difference Time-Domain Code, http://www.borg.umn.edu/toyfdtd/

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of California, Irvine, Irvine, CA, 92697, USA

    Sevin Fide & Stephen Jenks

Authors
  1. Sevin Fide
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Jenks
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sevin Fide.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fide, S., Jenks, S. A middleware approach for pipelining communications in clusters. Cluster Comput 10, 409–424 (2007). https://doi.org/10.1007/s10586-007-0026-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-007-0026-7

Keywords

Search

Navigation