Abstract
The InfiniBand architecture is among the leading interconnects that support high performance computing. The high bandwidth and low latency provided by InfiniBand are increasing its applications outside the high performance computing domain. One of the important application domains is the critical infrastructure.
However, InfiniBand is not immune to security risks. Previous research has shown that common traffic analysis tools cannot effectively monitor InfiniBand traffic transmitted between hosts. This is due to the kernel bypass nature of the InfiniBand architecture and remote direct memory access operations. However, if the Remote Direct Memory Access over Converged Ethernet (RoCE) protocol is employed, it is possible to restore traffic visibility in novel ways. This research demonstrates that the approach, coupled with an InfiniBand-capable adapter, enables common traffic analysis tools to be used to monitor InfiniBand network traffic without sacrificing bandwidth and performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
J. Corbet, A. Rubini and G. Kroah-Hartman, Linux Device Drivers, O’Reilly Media, Sebastopol, California, 2005.
N. Dandapanthula, H. Subramoni, J. Vienne, K. Kandalla, S. Sur, D. Panda and R. Brightwell, INAM - A scalable InfiniBand network analysis and monitoring tool, in Euro-Par 2011: Parallel Processing Workshops, Part II, M. Alexander, P. D’Ambra, A. Belloum, G. Bosilca, M. Cannataro, M. Danelutto, B. Di Martino, M. Gerndt, E. Jeannot, R. Namyst, J. Roman, S. Scott, J. Traff, G. Vallee and J. Weidendorfer (Eds.), Springer, Berlin Heidelberg, Germany, pp. 166–177, 2011.
J. Hennessy and D. Patterson, Computer Architecture: A Quantitative Approach, Morgan Kaufmann, San Francisco, California, 2011.
InfiniBand Trade Association, InfiniBand Architecture Specification, Volume 1, Release 1.4, Beaverton, Oregon, 2020.
H. Jin, T. Cortes and R. Buyya (Eds.), High Performance Mass Storage and Parallel I/O: Technologies and Applications, Wiley-IEEE Press, New York, 2001.
M. Lee and E. Kim, A comprehensive framework for enhancing security in the InfiniBand architecture, IEEE Transactions on Parallel and Distributed Systems, vol. 18(10), pp. 1393–1406, 2007.
M. Lee, E. Kim and M. Yousif, Security enhancement in the InfiniBand architecture, Proceedings of the Nineteenth IEEE International Parallel and Distributed Processing Symposium, 2005.
Linux Foundation, What is Open vSwitch? San Francisco, California (docs.openvswitch.org/en/latest/intro/what-is-ovs), 2016.
P. MacArthur and R. Russell, A performance study to guide RDMA programming decisions, Proceedings of the Fourteenth IEEE International Conference on High Performance Computing and Communications and the Ninth IEEE International Conference on Embedded Software and Systems, pp. 778–785, 2012.
Mellanox Technologies, Introduction to InfiniBand, White Paper, Document No. 2003WP, Santa Clara, California (www.mellanox.com/pdf/whitepapers/IB_Intro_WP_190.pdf), 2003.
Mellanox Technologies, InfiniBand Software and Protocols Enable Seamless Off-the-Shelf Applications Deployment, White Paper, Sunnyvale, California (www.mellanox.com/pdf/whitepapers/WP_2007_IB_Software_and_Protocols.pdf), 2007.
Mellanox Technologies, RDMA Aware Networks Programming User Manual, Rev. 1.7, Sunnyvale, California (www.mellanox.com/related-docs/prod_software/RDMA_Aware_Programming_user_manual.pdf), 2015.
Mellanox Technologies, BlueField SmartNIC Modes, Sunnyvale, California (community.mellanox.com/s/article/BlueField-SmartNIC-Modes), 2019.
Mellanox Technologies, Nvidia Mellanox BlueField SmartNIC for InfiniBand and Ethernet, Sunnyvale, California (www.mellanox.com/files/doc-2020/pb-bluefield-vpi-smart-nic.pdf), 2020.
L. Mireles, S. Graham, S. Dunlap, P. Sweeney and M. Dallmeyer, Securing an InfiniBand network and its effect on performance, in Critical Infrastructure Protection XIV, J. Staggs and S. Shenoi (Eds.), Springer, Cham, Switzerland, pp. 157–179, 2020.
ntop, Vanilla PF_RING, Pisa, Italy (www.ntop.org/guides/pf_ring/vanilla.html), 2018.
D. Schmitt, S. Graham, P. Sweeney and R. Mills, Vulnerability assessment of InfiniBand networking, in Critical Infrastructure Protection XIII, J. Staggs and S. Shenoi (Eds.), Springer, Cham, Switzerland, pp. 179–205, 2019.
E. Strohmaier, J. Dongarra, H. Simon and M. Meuer, Top 500 The List, Prometeus, Sinsheim, Germany, 2020.
K. Subedi, D. Dasgupta and B. Chen, Security analysis of InfiniBand protocol implementations, Proceedings of the IEEE Symposium Series on Computational Intelligence, 2016.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply
About this paper
Cite this paper
Hintze, K., Graham, S., Dunlap, S., Sweeney, P. (2022). InfiniBand Network Monitoring: Challenges and Possibilities. In: Staggs, J., Shenoi, S. (eds) Critical Infrastructure Protection XV. ICCIP 2021. IFIP Advances in Information and Communication Technology, vol 636. Springer, Cham. https://doi.org/10.1007/978-3-030-93511-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-93511-5_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93510-8
Online ISBN: 978-3-030-93511-5
eBook Packages: Computer ScienceComputer Science (R0)
