research-article

Your Programmable NIC Should be a Programmable Switch

Authors:

Brent Stephens,

Michael M. SwiftAuthors Info & Claims

HotNets '18: Proceedings of the 17th ACM Workshop on Hot Topics in Networks

Pages 36 - 42

https://doi.org/10.1145/3286062.3286068

Published: 15 November 2018 Publication History

Abstract

Today's NICs are becoming programmable ("smart"). To support new network protocols, services, and offloads, there are NICs today that have on-board FPGAs, embedded processors, programmable forwarding pipelines, and specialized engines to support features like RDMA. Unfortunately, existing programmable NICs have a number of key limitations. It is difficult to chain offloads, schedule competing accesses to shared resources, and support functions that require variable processing time and thus may not run at line-rate.

In this paper, we propose PANIC, a new architecture for programmable NICs that overcomes the limitations of existing NIC designs. We divide the NIC into three components: 1) self-contained offload engines, 2) a logical switch, and 3) a logical scheduler. This design overcomes the limitations of existing designs and is able to scale with increasing line-rates to a large number of offloads and long offload chains.

Supplementary Material

MP4 File (p36-stephens.mp4)

Download
1042.38 MB

References

[1]

Intel ethernet switch fm10000 datasheet. https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/ethernet-multi-host-controller-fm10000-family-datasheet.pdf.

[2]

Accolade Technology. Accolade ANIC. https://accoladetechnology.com/whitepapers/ANIC-Features-Overview.pdf.

[3]

Barefoot. Barefoot Tofino. https://www.barefootnetworks.com/technology/#tofino, 2017.

[4]

P. Bosshart, D. Daly, G. Gibb, M. Izzard, N. McKeown, J. Rexford, C. Schlesinger, D. Talayco, A. Vahdat, G. Varghese, and D. Walker. P4: Programming protocol-independent packet processors. SIGCOMM CCR, 2014.

Digital Library

[5]

P. Bosshart, G. Gibb, H.-S. Kim, G. Varghese, N. McKeown, M. Izzard, F. A. Mujica, and M. Horowitz. Forwarding metamorphosis: fast programmable match-action processing in hardware for SDN. In SIGCOMM. ACM, 2013.

Digital Library

[6]

L. D. Carli, Y. Pan, A. Kumar, C. Estan, and K. Sankaralingam. PLUG: Flexible lookup modules for rapid deployment of new protocols in high-speed routers. In SIGCOMM. ACM, 2009.

Digital Library

[7]

Cavium Corporation. Cavium CN63XX-NIC10E. http://cavium.com/Intelligent_Network_Adapters_CN63XX_NIC10E.html.

[8]

Cavium Corporation. Cavium LiquidIO. http://www.cavium.com/pdfFiles/LiquidIO_Server_Adapters_PB_Rev1.2.pdf.

[9]

S. Chole, A. Fingerhut, S. Ma, A. Sivaraman, S. Vargaftik, A. Berger, G. Mendelson, M. Alizadeh, S.-T. Chuang, I. Keslassy, A. Orda, and T. Edsall. dRMT: Disaggregated programmable switching. In SIGCOMM. ACM, 2017.

Digital Library

[10]

W. Dally and B. Towles. Principles and Practices of Interconnection Networks. Morgan Kaufmann Publishers Inc., 2003.

Digital Library

[11]

W. J. Dally. Performance analysis of k-ary n-cube interconnection networks. IEEE Trans. Comput., 39(6), June 1990.

Digital Library

[12]

Exablaze. ExaNIC V5P. https://exablaze.com//exanic-v5p.

[13]

D. Firestone, A. Putnam, S. Mundkur, D. Chiou, A. Dabagh, M. Andrewartha, H. Angepat, V. Bhanu, A. Caulfield, E. Chung, H. K. Chandrappa, S. Chaturmohta, M. Humphrey, J. Lavier, N. Lam, F. Liu, K. Ovtcharov, J. Padhye, G. Popuri, S. Raindel, T. Sapre, M. Shaw, G. Silva, M. Sivakumar, N. Srivastava, A. Verma, Q. Zuhair, D. Bansal, D. Burger, K. Vaid, D. A. Maltz, and A. Greenberg. Azure accelerated networking: SmartNICs in the public cloud. In NSDI. USENIX Association, 2018.

[14]

M. Flajslik and M. Rosenblum. Network interface design for low latency request-response protocols. In USENIX ATC, 2013.

Digital Library

[15]

Intel. Intel 82599 10 GbE controller datasheet. http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/82599-10-gbe-controller-datasheet.pdf.

[16]

X. Jin, X. Li, H. Zhang, R. Soulé, J. Lee, N. Foster, C. Kim, and I. Stoica. NetCache: Balancing key-value stores with fast in-network caching. SOSP. ACM, 2017.

Digital Library

[17]

A. Kaufmann, S. Peter, N. K. Sharma, T. Anderson, and A. Krishnamurthy. High performance packet processing with FlexNIC. In ASPLOS. ACM, 2016.

Digital Library

[18]

Y. Le, H. Chang, S. Mukherjee, L. Wang, A. Akella, M. M. Swift, and T. V. Lakshman. UNO: Uniflying host and smart NIC offload for flexible packet processing. In SoCC. ACM, 2017.

Digital Library

[19]

J. Li, E. Michael, N. K. Sharma, A. Szekeres, and D. R. K. Ports. Just say NO to paxos overhead: Replacing consensus with network ordering. In OSDI. USENIX, 2016.

Digital Library

[20]

J. Li, E. Michael, N. K. Sharma, A. Szekeres, and D. R. K. Ports. Just say NO to paxos overhead: Replacing consensus with network ordering. In OSDI. USENIX, 2016.

Digital Library

[21]

M. Liu, L. Luo, J. Nelson, L. Ceze, A. Krishnamurthy, and K. Atreya. Incbricks: Toward in-network computation with an in-network cache. In ASPLOS. ACM, 2017.

Digital Library

[22]

Mellanox Technologies. Innova - 2 Flex Programmable Network Adapter. http://www.mellanox.com/related-docs/prod_adapter_cards/PB_Innova-2_Flex.pdf.

[23]

Mellanox Technologies. Mellanox BlueField SmartNIC. http://www.mellanox.com/related-docs/prod_adapter_cards/PB_BlueField_Smart_NIC.pdf.

[24]

R. Miao, H. Zeng, C. Kim, J. Lee, and M. Yu. SilkRoad: Making stateful layer-4 load balancing fast and cheap using switching ASICs. In SIGCOMM. ACM, 2017.

Digital Library

[25]

R. Mittal, R. Agarwal, S. Ratnasamy, and S. Shenker. Universal packet scheduling. In NSDI. USENIX, 2016.

Digital Library

[26]

J. C. Mogul. TCP offload is a dumb idea whose time has come. In HotOS. USENIX, 2003.

Digital Library

[27]

R. Narayanan, S. Kotha, G. Lin, A. Khan, S. Rizvi, W. Javed, H. Khan, and S. A. Khayam. Macroflows and microflows: Enabling rapid network innovation through a split SDN data plane. In EWSDN. IEEE, 2012.

Digital Library

[28]

Netronome. NFP-6xxx flow processor. https://netronome.com/product/nfp-6xxx/.

[29]

S. Radhakrishnan, Y. Geng, V. Jeyakumar, A. Kabbani, G. Porter, and A. Vahdat. SENIC: Scalable NIC for end-host rate limiting. In NSDI, 2014.

Digital Library

[30]

K. K. Ram, J. Mudigonda, A. L. Cox, S. Rixner, P. Ranganathan, and J. R. Santos. sNICh: Efficient last hop networking in the data center. In ANCS. ACM/IEEE, 2010.

Digital Library

[31]

N. K. Sharma, M. Liu, K. Atreya, and A. Krishnamurthy. Approximating fair queueing on reconfigurable switches. In NSDI. USENIX, 2018.

[32]

P. Shinde, A. Kaufmann, K. Kourtis, and T. Roscoe. Modeling NICs with Unicorn. In PLOS. ACM, 2013.

Digital Library

[33]

P. Shinde, A. Kaufmann, T. Roscoe, and S. Kaestle. We need to talk about NICs. In HotOS. USENIX, 2013.

Digital Library

[34]

A. Sivaraman, A. Cheung, M. Budiu, C. Kim, M. Alizadeh, H. Balakrishnan, G. Varghese, N. McKeown, and S. Licking. Packet transactions: High-level programming for line-rate switches. In SIGCOMM, SIGCOMM, 2016.

Digital Library

[35]

A. Sivaraman, S. Subramanian, M. Alizadeh, S. Chole, S.-T. Chuang, A. Agrawal, H. Balakrishnan, T. Edsall, S. Katti, and N. McKeown. Programmable packet scheduling at line rate. In SIGCOMM. ACM, 2016.

Digital Library

[36]

S. Sivasubramanian. Amazon dynamoDB: A seamlessly scalable non-relational database service. In SIGMOD. ACM, 2012.

Digital Library

[37]

N. Sultana, S. Galea, D. Greaves, M. Wojcik, J. Shipton, R. Clegg, L. Mai, P. Bressana, R. Soulé, R. Mortier, P. Costa, P. Pietzuch, J. Crowcroft, A. W. Moore, and N. Zilberman. Emu: Rapid prototyping of networking services. In USENIX ATC. USENIX, 2017.

Digital Library

[38]

Tilera. Tile Processor Architecture Overview For the TILE-GX Series. http://www.mellanox.com/repository/solutions/tile-scm/docs/UG130-ArchOverview-TILE-Gx.pdf.

[39]

D. Wentzlaff, P. Griffin, H. Hoffmann, L. Bao, B. Edwards, C. Ramey, M. Mattina, C.-C. Miao, J. F. Brown III, and A. Agarwal. On-chip interconnection architecture of the tile processor. IEEE Micro, 27(5), Sept. 2007.

Digital Library

[40]

Y. Zhang, J. Gu, Y. Lee, M. Chowdhury, and K. G. Shin. Performance isolation anomalies in RDMA. In KBNets. ACM, 2017.

Digital Library

[41]

Y. Zhu, H. Eran, D. Firestone, C. Guo, M. Lipshteyn, Y. Liron, J. Padhye, S. Raindel, M. H. Yahia, and M. Zhang. Congestion control for large-scale RDMA deployments. In SIGCOMM. ACM, 2015.

Digital Library

Cited By

Carvalho FFerreira RCunha ÍVieira MRamanathan M(2024)State Disaggregation for Dynamic Scaling of Network FunctionsIEEE/ACM Transactions on Networking10.1109/TNET.2023.328256232:1(81-95)Online publication date: Feb-2024
https://doi.org/10.1109/TNET.2023.3282562
Chang BSubramanian KD'Antoni LAkella A(2024)Learned load balancingTheoretical Computer Science10.1016/j.tcs.2024.1146111003:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.tcs.2024.114611
Yao RZhang ZFang GGao PLiu SFan YXu YChao HSchulzrinne HKohler EMaltz DMisra V(2023)BMW Tree: Large-scale, High-throughput and Modular PIFO Implementation using Balanced Multi-Way Sorting TreeProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604862(208-219)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604862
Show More Cited By

Recommendations

Embedded Design Using Programmable Gate Arrays
A programmable and scalable openflow switch using heterogeneous soc platforms
HotSDN '14: Proceedings of the third workshop on Hot topics in software defined networking

This work presents a hardware-software co-design approach of an OpenFlow switch using a state-of-the-art heterogeneous System-on-chip (SoC) platform. Specifically, we implement the OpenFlow switch on a Xilinx Zynq ZC706 board. The Xilinx Zynq SoC family ...
Field Programmable Gate Array (FPGA) and Their Applications

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HotNets '18: Proceedings of the 17th ACM Workshop on Hot Topics in Networks

November 2018

191 pages

ISBN:9781450361200

DOI:10.1145/3286062

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

HotNets '18

Sponsor:

SIGCOMM

HotNets '18: The 17th ACM workshop on Hot Topics in Networks

November 15 - 16, 2018

WA, Redmond, USA

Acceptance Rates

Overall Acceptance Rate 110 of 460 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
751
Total Downloads

Downloads (Last 12 months)61
Downloads (Last 6 weeks)9

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Carvalho FFerreira RCunha ÍVieira MRamanathan M(2024)State Disaggregation for Dynamic Scaling of Network FunctionsIEEE/ACM Transactions on Networking10.1109/TNET.2023.328256232:1(81-95)Online publication date: Feb-2024
https://doi.org/10.1109/TNET.2023.3282562
Chang BSubramanian KD'Antoni LAkella A(2024)Learned load balancingTheoretical Computer Science10.1016/j.tcs.2024.1146111003:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.tcs.2024.114611
Yao RZhang ZFang GGao PLiu SFan YXu YChao HSchulzrinne HKohler EMaltz DMisra V(2023)BMW Tree: Large-scale, High-throughput and Modular PIFO Implementation using Balanced Multi-Way Sorting TreeProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604862(208-219)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604862
Gootzen PPfefferle JStoica RTrivedi AGilad YKostic DMoatti YBiran O(2023)DPFS: DPU-Powered File System VirtualizationProceedings of the 16th ACM International Conference on Systems and Storage10.1145/3579370.3594769(1-7)Online publication date: 5-Jun-2023
https://dl.acm.org/doi/10.1145/3579370.3594769
Chang BAkella AD'Antoni LSubramanian K(2023)Learned Load BalancingProceedings of the 24th International Conference on Distributed Computing and Networking10.1145/3571306.3571403(177-187)Online publication date: 4-Jan-2023
https://dl.acm.org/doi/10.1145/3571306.3571403
Lin JGuo ZChen X(2022)MAToC: A Novel Match-Action Table Architecture on Corundum for 8 × 25G NetworkingApplied Sciences10.3390/app1217873412:17(8734)Online publication date: 31-Aug-2022
https://doi.org/10.3390/app12178734
Simpson KPezaros D(2022)Revisiting the Classics: Online RL in the Programmable DataplaneNOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium10.1109/NOMS54207.2022.9789930(1-10)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1109/NOMS54207.2022.9789930
Pekar ADuque‐Torres ASeah WCaicedo Rendon O(2022)Towards threshold‐agnostic heavy‐hitter classificationInternational Journal of Network Management10.1002/nem.218832:3Online publication date: 7-Apr-2022
https://dl.acm.org/doi/10.1002/nem.2188
Schuh HLiang WLiu MNelson JKrishnamurthy A(2021)XenicProceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles10.1145/3477132.3483555(740-755)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3477132.3483555
Wolnikowski AIbanez SStone JKim CManohar RSoulé RAngel SKasikci BKohler E(2021)ZerializerProceedings of the Workshop on Hot Topics in Operating Systems10.1145/3458336.3465283(206-212)Online publication date: 1-Jun-2021
https://dl.acm.org/doi/10.1145/3458336.3465283
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten