Skip to main content
SpringerLink
Log in

High-performance message striping over reliable transport protocols

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This paper introduces a high-performance middleware-level message striping approach to increase communication bandwidth for data transfer in heterogeneous clusters equipped with multiple networks. In this scheme, concurrency is used for the striping process. The proposed striping approach is designed to work at the middleware-level, between the distributed applications and the reliable transport protocols such as TCP. The middleware-level striping approach provides flexible, scalable, and hardware-, network-, and operating systems-independent communication bandwidth solution. In addition, techniques to enhance the performance of this approach over multiple networks are introduced. The proposed techniques, which minimize synchronization contention and eliminate the striping sequence header, rely on the features of a reliable transport protocol such as TCP to reduce some of the concurrent striping overhead. The techniques have been implemented and evaluated on a real cluster with multiple networks and the results show significant performance gains for data transfer over existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adiseshu H, Parulkar G, Vargese G (1996) A reliable and scalable striping protocol. Comp. Communication Rev. 26:131–141

    Article  Google Scholar 

  2. Beowulf Ethernet Channel bonding web page at http://www.beowulf.org/software/bonding.html, June 2002.

  3. Configuring Ethernet Channel bonding, http://www.beowulf-underground.org/doc_project/BIAA-HOWTO/Beowulf-Installation-and-Administration-HOWTO-12.html

  4. Brendan C, Traw S, Smith J (1995) Striping within the network subsystem. IEEE Netw. 22–29

  5. Channel bonding benchmark results at http://www.fos.su.se/compchem/jazz/bond.html, June 2002.

  6. Dongarra J, Hey T, Strohmaier E (1996) Selected results from the PARKBENCH benchmark. In Proceedings of the 2nd European Conference on Parallel Processing (Euro-Par ’96), pp. 251–254,

  7. Figueira S, Berman F (2001) A slowdown model for applications executing on time-shared clusters of workstations. IEEE Transactions on Parallel and Distributed Systems 12(6):653–670

    Article  Google Scholar 

  8. Getov V, Hernandez E, Hey T (1997) Message-passing performance of parallel computers. In Proceedings of the 3rd European Conference on Parallel Processing (Euro-Par ’97) pp. 1,009–1,016,

  9. GridFTP: Universal Data Transfer for the Grid, The Globus Project white paper. 2000. The University of Chicago and the University of Southern California. At http://www.globus.org/datagrid/deliverables/C2WPdraft3.pdf, and GridFTP Update. 2002. At http://www.globus.org/datagrid/deliverables/GridFTP-Overview-200201.pdf

  10. Katz R, Gibson G, Patterson D (1989) Disk system architectures for high performance computing. In Proc. of the IEEE 77(12):1842–1858,

    Article  Google Scholar 

  11. Mohamed N, Al-Jaroodi J, Jiang H, Swanson D (2002) A user-level socket layer over multiple physical network interfaces. In Proceedings of the 14th International Conference on Parallel and Distributed Computing and Systems pp. 810–815,

  12. Mohamed N, Al-Jaroodi J, Jiang H, Swanson D (2003) Performance properties of combined heterogeneous networks. In the Proceedings of IPDPS 2003, International Workshop on Performance Modeling, Evaluation, and Optimization of Parallel and Distributed Systems (PMEO-PDS’03), IEEE,

  13. Mohamed N, Al-Jaroodi J, Jiang H, Swanson D (2003) A Middleware-Level Parallel Transfer Technique over Multiple Network Interfaces. ClusterWorld Conference and Expo, San Jose, California

  14. Mohamed N, Al-Jaroodi J, Jiang H, Swanson D (2003) Scalable bulk data transfer in wide area networks. International Journal of High Performance Computing Applications 17(3)

  15. Theoharakis V, Guerin R (1993) SONET OD-12 interface for variable length packets. The Second International Conference On Computer Communication and Networks

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Department, Stevens Institute of Technology, Hoboken, NJ, 07030

    Nader Mohamed & Jameela Al-Jaroodi

  2. Department of Computer Science and Engineering, University of Nebraska - Lincoln, Lincoln, NE, 68588-0115

    Hong Jiang & David Swanson

Authors
  1. Nader Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jameela Al-Jaroodi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Swanson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jameela Al-Jaroodi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mohamed, N., Al-Jaroodi, J., Jiang, H. et al. High-performance message striping over reliable transport protocols. J Supercomput 38, 261–278 (2006). https://doi.org/10.1007/s11227-006-8443-6

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-006-8443-6

Keywords

Search

Navigation