Skip to main content
SpringerLink
Log in

Sockets Direct Protocol for Hybrid Network Stacks: A Case Study with iWARP over 10G Ethernet

  • Conference paper
High Performance Computing - HiPC 2008 (HiPC 2008)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 5374))

Included in the following conference series:

  • 699 Accesses

Abstract

As high-end computing systems continue to grow, the need for advanced networking capabilities, such as hot-spot avoidance and fault tolerance, is becoming important. While the traditional approach of utilizing intelligent network hardware has worked well to achieve high performance, adding more and more features makes the hardware complex and expensive. Consequently, protocol stacks such as iWARP and MX for 10-Gigabit Ethernet and QLogic InfiniBand, utilize hybrid hardware-software designs that take advantage of the processing power of multi-core processors together with network hardware accelerators. However, upper-layer stacks on these networks, such as the Sockets Direct Protocol (SDP), have not kept pace with such shift in paradigm, and have continued to assume complete hardware offload, leading to redundant features and performance loss. In this paper, we propose an enhanced design for SDP that allows network stacks to specify components implemented in hardware and software, and uses this information to optimize its execution.

This work was supported in part by the National Science Foundation Grant #0702182 and the Mathematical, Information, and Computational Sciences Division subprogram of the Office of Advanced Scientific Computing Research, Office of Science, U.S. Department of Energy, under Contract DE-AC02-06CH11357.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. IETF, http://www.ietf.org

  2. Mellanox Technologies, http://www.mellanox.com

  3. RDMA Consortium, http://www.rdmaconsortium.org

  4. Afework, A., Beynon, M.D., Bustamante, F., Demarzo, A., Ferreira, R., Miller, R., Silberman, M., Saltz, J., Sussman, A., Tsang, H.: Digital Dynamic Telepathology - The Virtual Microscope. In: Proceedings of the 1998 AMIA Annual Fall Symposium, American Medical Informatics Association (November 1998)

    Google Scholar 

  5. AMD Quad-core Opteron processor, http://multicore.amd.com/us-en/quadcore/

  6. Bailey, S., Talpey, T.: Remote Direct Data Placement (RDDP) (April 2005)

    Google Scholar 

  7. Balaji, P., Bhagvat, S., Panda, D.K., Thakur, R., Gropp, W.: Advanced Flow-control Mechanisms for the Sockets Direct Protocol over InfiniBand. In: ICPP (2007)

    Google Scholar 

  8. Balaji, P., Feng, W., Bhagvat, S., Panda, D.K., Thakur, R., Gropp, W.: Analyzing the Impact of Supporting Out-of-Order Communication on In-order Performance with iWARP. In: Lumpe, M., Vanderperren, W. (eds.) SC 2007. LNCS, vol. 4829. Springer, Heidelberg (2007)

    Google Scholar 

  9. Balaji, P., Jin, H.W., Vaidyanathan, K., Panda, D.K.: Supporting iWARP Compatibility and Features for Regular Network Adapters. In: RAIT (2005)

    Google Scholar 

  10. Balaji, P., Narravula, S., Vaidyanathan, K., Krishnamoorthy, S., Wu, J., Panda, D.K.: Sockets Direct Protocol over InfiniBand in Clusters: Is it Beneficial? In: ISPASS 2004 (2004)

    Google Scholar 

  11. Beynon, M., Kurc, T., Sussman, A., Saltz, J.: Design of a framework for data-intensive wide-area applications. In: HCW (2000)

    Google Scholar 

  12. Beynon, M.D., Kurc, T., Catalyurek, U., Chang, C., Sussman, A., Saltz, J.: Distributed Processing of Very Large Datasets with DataCutter. In: Parallel Computing (October 2001)

    Google Scholar 

  13. Beynon, M.D., Kurc, T., Catalyurek, U., Saltz, J.: A Component-based Implementation of Iso-surface Rendering for Visualizing Large Datasets. Report CS-TR-4249 and UMIACS-TR-2001-34, University of Maryland, Department of Computer Science and UMIACS (2001)

    Google Scholar 

  14. Boden, N.J., Cohen, D., Felderman, R.E., Kulawik, A.E., Seitz, C.L., Seizovic, J.N., Su, W.K.: Myrinet: A Gigabit-per-Second Local Area Network. In: IEEE Micro 1995 (1995)

    Google Scholar 

  15. Boppana, R.V., Chalasani, S.: Fault-Tolerant Wormhole Routing Algorithms for Mesh Networks. IEEE Transactions on Computers, 848–864 (July 1995)

    Google Scholar 

  16. Herrmann, S., Castagnoli, M., Brauer, G.: Optimization of cyclic redundancy-check codes with 24 and 32 paritybits. IEEE Transactions on Communication (1993)

    Google Scholar 

  17. Catalyurek, U., Beynon, M.D., Chang, C., Kurc, T., Sussman, A., Saltz, J.: The Virtual Microscope. IEEE Transactions on Information Technology in Biomedicine (to appear, 2002)

    Google Scholar 

  18. Cohen, A., Rangarajan, S., Slye, H.: On the Performance of TCP Splicing for URL-aware Redirection. In: USENIX 1999 (1999)

    Google Scholar 

  19. Chelsio Communications, http://www.chelsio.com

  20. Culley, P., Elzur, U., Recio, R., Bailey, S.: Marker PDU Aligned Framing for TCP Specification (November 2002)

    Google Scholar 

  21. Dalessandro, D., Devulapalli, A., Wyckoff, P.: Design and Implementation of the iWARP Protocol in Software. In: PDCS 2005 (2005)

    Google Scholar 

  22. Gao, J., Shen, H.: Parallel view dependent isosurface extraction using multi-pass occlusion culling. In: ACM SIGGRAPH (2001)

    Google Scholar 

  23. NetEffect Inc., http://www.neteffect.com/product-features.html

  24. InfiniBand Trade Association, http://www.infinibandta.org/

  25. Intel Core 2 Extreme quad-core processor, http://www.intel.com/products/processor/core2xe/qc_prod_brief.pdf

  26. Khosravi, H.M., Foong, A.: Performance Analysis of iSCSI and Effect of CRC Computation. In: BEACON 2004 (2004)

    Google Scholar 

  27. Myricom. Myrinet home page, http://www.myri.com/

  28. Petrini, F., Feng, W.C., Hoisie, A., Coll, S., Frachtenberg, E.: The Quadrics Network (QsNet): High-Performance Clustering Technology. In: Hot Interconnects (2001)

    Google Scholar 

  29. Pfister, G.F., Norton, V.A.: Hot-spot Contention and Combining in Multistage Interconnection Networks. IEEE Transactions on Computers 34, 943–948 (1985)

    Article  Google Scholar 

  30. Qlogic Corporation, http://www.qlogic.com

  31. Recio, R., Culley, P., Garcia, D., Hilland, J., Metzler, B.: An RDMA protocol specification (April 2005), http://www.ietf.org/internet-drafts/draft-ietf-rddp-rdmap-04.txt

  32. Sarvate, D.V.: Computation of cyclic redundancy checks via table look-up. Communications of the ACM 31 (1998)

    Google Scholar 

  33. Vishnu, A., Koop, M., Moody, A., Mamidala, A., Narravula, S., Panda, D.K.: Hot-Spot Avoidance With Multi-Pathing Over InfiniBand: An MPI Perspective. In: CCGrid (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Math. and Comp. Sci., Argonne Natl. Lab, USA

    Pavan Balaji & Rajeev Thakur

  2. Scalable Systems Group, Dell Inc., USA

    Sitha Bhagvat

  3. Computer Science and Engg., Ohio State University, USA

    Dhabaleswar K. Panda

Authors
  1. Pavan Balaji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sitha Bhagvat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajeev Thakur
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dhabaleswar K. Panda
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, The Ohio State University, 2015 Neil Avenue, OH 43210, Columbus, USA

    Ponnuswamy Sadayappan

  2. Department of Electrical and Computer Engineering, Rutgers, the State University of New Jersey, 94 Brett Road, NJ 08854, Piscataway, USA

    Manish Parashar

  3. Hewlett-Packard ISO,, Sy 192, Whitefield Road, Mahadevapura Post, 560048, Bangalore, India

    Ramamurthy Badrinath

  4. Department of Electrical Engineering, University of Southern California, CA 90089-2562, Los Angeles, USA

    Viktor K. Prasanna

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Balaji, P., Bhagvat, S., Thakur, R., Panda, D.K. (2008). Sockets Direct Protocol for Hybrid Network Stacks: A Case Study with iWARP over 10G Ethernet. In: Sadayappan, P., Parashar, M., Badrinath, R., Prasanna, V.K. (eds) High Performance Computing - HiPC 2008. HiPC 2008. Lecture Notes in Computer Science, vol 5374. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89894-8_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-89894-8_42

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-89893-1

  • Online ISBN: 978-3-540-89894-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation