Abstract
In the context of large-scale datacenter networks, the distinct mechanisms of these two protocols result in mutual interference when TCP and RoCE flows traverse shared segments of network. TCP may preempt the bandwidth allocated to RoCE, leading to unfair allocation of shared buffers. Nevertheless, due to the cost trade-off, the consideration of shared buffers becomes necessary. Therefore, isolating through traffic classes does not effectively address the issue. To meet the challenge, in this paper, we present BufferConcede, a switch-based traffic control method that allows TCP traffic to occupy more buffer space in non-hybrid switch, thereby weakening TCP traffic’s greediness for buffer space. We integrate it with practical congestion control approaches such as DCQCN, DCTCP. We evaluate BufferConcede in both implementation and large-scale simulations. Compared with state of the act, BufferConcede can reduce bandwidth allocation errors under 2.0% and reduce the average FCT of RoCE traffic by approximately 4.8%–10.2%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wu, H., Feng, Z., Guo, C., Zhang, Y.: ICTCP: incast congestion control for TCP in data center networks. In: Proceedings of the 6th International Conference, pp. 1–12. (2010)
Alizadeh, M.: Data center TCP (DCTCP). In: Proceedings of the ACM SIGCOMM 2010 Conference, pp. 63–74 (2010)
Guo, C.: RDMA over commodity ethernet at scale. In: Proceedings of the ACM SIGCOMM 2016 Conference, pp. 202–215 (2016)
Karol, M., Golestani, S.J., Lee, D.: Prevention of deadlocks and livelocks in lossless backpressured packet networks. IEEE/ACM Trans. Network. 11(6), 923–934 (2003)
Das, S., Sankar, R.: Broadcom smart-buffer technology in data center switches for cost-effective performance scaling of cloud applications (2020). https://docs.broadcom.com/docs-and-downloads/collateral/etp/SBT-ETP100.pdf
Linux Foundation 2015. Data Plane Development Kit (DPDK). http://www.dpdk.org. (2015)
Zhu, Y., et al.: Congestion control for large-scale RDMA deployments. In: Proceedings of the ACM SIGCOMM 2015 Conference, pp. 523–536 (2015)
Li, Y., et al.: HPCC: High precision congestion control. In: Proceedings of the ACM SIGCOMM 2019 Conference, pp. 44–58 (2019)
Gao, P.X et al.: Network requirements for resource disaggregation. In: 12th USENIX Symposium on Operating Systems Design and Implementation, pp. 249–264 (2016)
Infiniband Trade Association. 2014. Supplement to InfiniBand architecture specification volume 1 release 1.2.2 annex A17: RoCEv2 (IP routable RoCE) (2014)
IEEE Std 802.1Q-2018, IEEE Standard for Local and Metropolitan Area Networks-Bridges and Bridged Networks
Gao, Y., et al.: When cloud storage meets RDMA. In: 18th USENIX Symposium on Networked Systems Design and Implementation, pp. 519–533 (2021)
Li, Q., et al.: More than capacity: performance-oriented evolution of Pangu in Alibaba. In: 21st USENIX Conference on File and Storage Technologies, pp. 331–346 (2023)
Yan, S., Wang, X., Zheng, X., Xia, Y., Liu, D., Deng, W.: ACC: automatic ECN tuning for high-speed datacenter networks. In: Proceedings of the 2021 ACM SIGCOMM 2021 Conference, pp. 384–397 (2021)
Olteanu, V., et al.: An edge-queued datagram service for all datacenter traffic. In: 19th USENIX Symposium on Networked Systems Design and Implementation, pp. 761–777 (2021)
Berkeley. Berkeley Extensible Software Switch (2018). https://github.com/NetSys/bess
Zhong, X., Zhang, J., Zhang, Y., Guan, Z., Wan, Z.: PACC: proactive and accurate congestion feedback for RDMA congestion control. In: IEEE INFOCOM 2022-IEEE Conference on Computer Communications, pp. 2228–2237. IEEE(2022)
Alibaba. 2019. HPCC simulator (2019). https://github.com/alibaba-edu/High-Precision-Congestion-Control
InfiniBand Architecture Volume 1 and released specification Volume 2 (2015). https://cw.infinibandta.org/document/dl/7859
IEEE. 802.11Qau. 2011. Priority based flow control (PFC)
Roy, A., Zeng, H., Bagga, J., Porter, G., Snoeren, A.C.: Inside the social network’s (datacenter) network. In: Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, pp. 123–137 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Meng, L. et al. (2025). BufferConcede: Conceding Buffer for RoCE Traffic in TCP/RoCE Mix-Flows. In: Cai, Z., Takabi, D., Guo, S., Zou, Y. (eds) Wireless Artificial Intelligent Computing Systems and Applications. WASA 2024. Lecture Notes in Computer Science, vol 14997. Springer, Cham. https://doi.org/10.1007/978-3-031-71464-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-71464-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-71463-4
Online ISBN: 978-3-031-71464-1
eBook Packages: Computer ScienceComputer Science (R0)