research-article

Generic External Memory for Switch Data Planes

Authors:

Srinivasan SeshanAuthors Info & Claims

HotNets '18: Proceedings of the 17th ACM Workshop on Hot Topics in Networks

Pages 1 - 7

https://doi.org/10.1145/3286062.3286063

Published: 15 November 2018 Publication History

Abstract

Network switches are an attractive vantage point to serve various network applications and functions such as load balancing and virtual switching because of their in-network location and high packet processing rate. Recent advances in programmable switch ASICs open more opportunities for offloading various functionality to switches. However, the limited memory capacity on switches has been a major challenge that such applications struggle to deal with. In this paper, we envision that by enabling network switches to access remote memory purely from data planes, the performance of a wide range of applications can be improved. We design three remote memory primitives, leveraging RDMA operations, and show the feasibility of accessing remote memory from switches using our prototype implementation.

Supplementary Material

MP4 File (p1-kim.mp4)

Download
1123.49 MB

References

[1]

2018. Advanced Network Telemetry. https://www.barefootnetworks.com/use-cases/ad-telemetry/.

[2]

2018. Barefoot Capilano. https://www.barefootnetworks.com/products/brief-capilano/.

[3]

2018. Barefoot Tofino. https://www.barefootnetworks.com/products/brief-tofino/.

[4]

2018. BCM88690--10 Tb/s StrataDNX Jericho2 Ethernet Switch Series. https://www.broadcom.com/products/ethernet-connectivity/switching/stratadnx/bcm88690.

[5]

2018. Cavium Xpliant Ethernet Switches. https://www.cavium.com/xpliant-ethernet-switch-product-family.html.

[6]

2018. netpipe(1) - Linux man page. https://linux.die.net/man/1/netpipe.

[7]

2018. Perftest package. https://github.com/linux-rdma/perftest.

[8]

Mohammad Alizadeh, Albert Greenberg, David A. Maltz, Jitendra Padhye, Parveen Patel, Balaji Prabhakar, Sudipta Sengupta, and Murari Sridharan. 2010. Data Center TCP (DCTCP). In ACM SIGCOMM (2010).

Digital Library

[9]

Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. 2014. P4: Programming Protocol-independent Packet Processors. SIGCOMM Comput. Commun. Rev. 44, 3 (July 2014), 87--95.

Digital Library

[10]

Yanzhe Chen, Xingda Wei, Jiaxin Shi, Rong Chen, and Haibo Chen. 2016. Fast and General Distributed Transactions Using RDMA and HTM. In EuroSys (2016).

Digital Library

[11]

Graham Cormode and Marios Hadjieleftheriou. 2008. Finding Frequent Items in Data Streams. Proc. VLDB Endow. 1, 2 (Aug. 2008), 1530--1541.

Digital Library

[12]

Aleksandar Dragojević, Dushyanth Narayanan, Miguel Castro, and Orion Hodson. 2014. FaRM: Fast Remote Memory. In USENIX NSDI (2014).

Digital Library

[13]

Aleksandar Dragojević, Dushyanth Narayanan, Edmund B. Nightingale, Matthew Renzelmann, Alex Shamis, Anirudh Badam, and Miguel Castro. 2015. No Compromises: Distributed Transactions with Consistency, Availability, and Performance. In ACM SOSP (2015).

Digital Library

[14]

Daniel Firestone, Andrew Putnam, Sambhrama Mundkur, Derek Chiou, Alireza Dabagh, Mike Andrewartha, Hari Angepat, Vivek Bhanu, Adrian Caulfield, Eric Chung, Harish Kumar Chandrappa, Somesh Chaturmohta, Matt Humphrey, Jack Lavier, Norman Lam, Fengfen Liu, Kalin Ovtcharov, Jitu Padhye, Gautham Popuri, Shachar Raindel, Tejas Sapre, Mark Shaw, Gabriel Silva, Madhan Sivakumar, Nisheeth Srivastava, Anshuman Verma, Qasim Zuhair, Deepak Bansal, Doug Burger, Kushagra Vaid, David A. Maltz, and Albert Greenberg. 2018. Azure Accelerated Networking: SmartNICs in the Public Cloud. In USENIX NSDI (2018).

[15]

IEEE. 2011. 802.1Qbb -- Priority-based Flow Control. https://1.ieee802.org/dcb/802-1qbb/.

[16]

Infiniband Trace Association. 2010. Supplement to InfiniBand architecture specification volume 1 release 1.2.1 annex A16: RDMA over converged ethernet (RoCE).

[17]

Infiniband Trace Association. 2010. Supplement to InfiniBand architecture specification volume 1 release 1.2.1 annex A17: RDMA over converged ethernet (RoCE).

[18]

Xin Jin, Xiaozhou Li, Haoyu Zhang, Nate Foster, Jeongkeun Lee, Robert Soulé, Changhoon Kim, and Ion Stoica. 2018. NetChain: Scale-Free Sub-RTT Coordination. In USENIX NSDI (2018).

Digital Library

[19]

Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica. 2017. NetCache: Balancing Key-Value Stores with Fast In-Network Caching. In ACM SOSP (2017).

Digital Library

[20]

Anuj Kalia, Michael Kaminsky, and David G. Andersen. 2014. Using RDMA Efficiently for Key-value Services. In ACM SIGCOMM (2014).

Digital Library

[21]

Daehyeok Kim, Amirsaman Memaripour, Anirudh Badam, Yibo Zhu, Hongqiang Harry Liu, Jitu Padhye, Shachar Raindel, Steven Swanson, Vyas Sekar, and Srinivasan Seshan. 2018. Hyperloop: Group-based NIC-offloading to Accelerate Replicated Transactions in Multi-tenant Storage Systems. In ACM SIGCOMM (2018).

Digital Library

[22]

Jialin Li, Ellis Michael, Naveen Kr. Sharma, Adriana Szekeres, and Dan R. K. Ports. 2016. Just Say No to Paxos Overhead: Replacing Consensus with Network Ordering. In USENIX OSDI (2016).

Digital Library

[23]

Zaoxing Liu, Antonis Manousis, Gregory Vorsanger, Vyas Sekar, and Vladimir Braverman. 2016. One Sketch to Rule Them All: Rethinking Network Flow Monitoring with UnivMon. In ACM SIGCOMM (2016).

Digital Library

[24]

Youyou Lu, Jiwu Shu, Youmin Chen, and Tao Li. 2017. Octopus: an RDMA-enabled Distributed Persistent Memory File System. In USENIX ATC (2017).

Digital Library

[25]

Nick McKeown, Tom Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson, Jennifer Rexford, Scott Shenker, and Jonathan Turner. 2008. OpenFlow: Enabling Innovation in Campus Networks. SIGCOMM Comput. Commun. Rev. 38, 2 (March 2008), 69--74.

Digital Library

[26]

Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. 2017. SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs. In ACM SIGCOMM (2017).

Digital Library

[27]

Christopher Mitchell, Yifeng Geng, and Jinyang Li. 2013. Using One-Sided RDMA Reads to Build a Fast, CPU-Efficient Key-Value Store. In USENIX ATC (2013).

Digital Library

[28]

Radhika Mittal, Vinh The Lam, Nandita Dukkipati, Emily Blem, Hassan Wassel, Monia Ghobadi, Amin Vahdat, Yaogong Wang, David Wetherall, and David Zats. 2015. TIMELY: RTT-based Congestion Control for the Datacenter. In ACM SIGCOMM (2015).

Digital Library

[29]

Srinivas Narayana, Anirudh Sivaraman, Vikram Nathan, Prateesh Goyal, Venkat Arun, Mohammad Alizadeh, Vimalkumar Jeyakumar, and Changhoon Kim. 2017. Language-Directed Hardware Design for Network Performance Monitoring. In ACM SIGCOMM (2017).

Digital Library

[30]

Ben Pfaff, Justin Pettit, Teemu Koponen, Ethan Jackson, Andy Zhou, Jarno Rajahalme, Jesse Gross, Alex Wang, Joe Stringer, Pravin Shelar, Keith Amidon, and Martin Casado. 2015. The Design and Implementation of Open vSwitch. In USENIX NSDI (2015).

Digital Library

[31]

Yizhou Shan, Shin-Yeh Tsai, and Yiying Zhang. 2017. Distributed Shared Persistent Memory. In ACM SoCC (2017).

Digital Library

[32]

Arjun Singh, Joon Ong, Amit Agarwal, Glen Anderson, Ashby Armistead, Roy Bannon, Seb Boving, Gaurav Desai, Bob Felderman, Paulie Germano, Anand Kanagala, Jeff Provost, Jason Simmons, Eiichi Tanda, Jim Wanderer, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2015. Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network. In ACM SIGCOMM (2015).

Digital Library

[33]

Minlan Yu, Jennifer Rexford, Michael J. Freedman, and Jia Wang. 2010. Scalable Flow-based Networking with DIFANE. In ACM SIGCOMM (2010).

Digital Library

[34]

Kyriakos Zarifis, Rui Miao, Matt Calder, Ethan Katz-Bassett, Minlan Yu, and Jitendra Padhye. 2014. DIBS: Justin-time Congestion Mitigation for Data Centers. In ACM EuroSys (2014).

Digital Library

[35]

Qiao Zhang, Vincent Liu, Hongyi Zeng, and Arvind Krishnamurthy. 2017. High-resolution Measurement of Data Center Microbursts. In ACM IMC (2017).

Digital Library

[36]

Yibo Zhu, Haggai Eran, Daniel Firestone, Chuanxiong Guo, Marina Lipshteyn, Yehonatan Liron, Jitendra Padhye, Shachar Raindel, Mohamad Haj Yahia, and Ming Zhang. 2015. Congestion Control for Large-Scale RDMA Deployments. In ACM SIGCOMM (2015).

Digital Library

Cited By

Li LXie KPei SWen JLiang WXie G(2024)CS-Sketch: Compressive Sensing Enhanced Sketch for Full Traffic MeasurementIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.330512511:3(2338-2352)Online publication date: May-2024
https://doi.org/10.1109/TNSE.2023.3305125
Luo SYu XLi KXing H(2024)Releasing the Power of In-Network Aggregation With Aggregator-Aware Routing OptimizationIEEE/ACM Transactions on Networking10.1109/TNET.2024.342338032:5(4488-4502)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3423380
Liu HChen XHuang QSun GWang PZhang DWu CLiu XYang Q(2024)Toward Resource-Efficient and High- Performance Program Deployment in Programmable NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2024.341338832:5(4270-4285)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3413388
Show More Cited By

Recommendations

A multiple shared memory switch
SSST '96: Proceedings of the 28th Southeastern Symposium on System Theory (SSST '96)

One of the problems that shared memory switches have is that in order to build large switches the memory access requirements become stringent. One way to overcome this is to utilize multiple buffers which space division multiplex the input lines and ...
Write-once-memory-code phase change memory
DATE '14: Proceedings of the conference on Design, Automation & Test in Europe

This paper describes a write-once-memory-code phase change memory (WOM-code PCM) architecture for next-generation non-volatile memory applications. Specifically, we address the long latency of the write operation in PCM --- attributed to PCM SET --- by ...
Enabling Hybrid PCM Memory System with Inherent Memory Management
RACS '16: Proceedings of the International Conference on Research in Adaptive and Convergent Systems

Replacing the traditional volatile main memory, e.g., DRAM, with a non-volatile phase change memory (PCM) has become a possible solution to reduce the energy consumption of computing systems. To further reduce the bit cost of PCM, the development trend ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HotNets '18: Proceedings of the 17th ACM Workshop on Hot Topics in Networks

November 2018

191 pages

ISBN:9781450361200

DOI:10.1145/3286062

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

HotNets '18

Sponsor:

SIGCOMM

HotNets '18: The 17th ACM workshop on Hot Topics in Networks

November 15 - 16, 2018

WA, Redmond, USA

Acceptance Rates

Overall Acceptance Rate 110 of 460 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
897
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)10

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li LXie KPei SWen JLiang WXie G(2024)CS-Sketch: Compressive Sensing Enhanced Sketch for Full Traffic MeasurementIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.330512511:3(2338-2352)Online publication date: May-2024
https://doi.org/10.1109/TNSE.2023.3305125
Luo SYu XLi KXing H(2024)Releasing the Power of In-Network Aggregation With Aggregator-Aware Routing OptimizationIEEE/ACM Transactions on Networking10.1109/TNET.2024.342338032:5(4488-4502)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3423380
Liu HChen XHuang QSun GWang PZhang DWu CLiu XYang Q(2024)Toward Resource-Efficient and High- Performance Program Deployment in Programmable NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2024.341338832:5(4270-4285)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3413388
Sun HHuang QLee PBai WZhu FBao Y(2024)Distributed Network Telemetry With Resource Efficiency and Full AccuracyIEEE/ACM Transactions on Networking10.1109/TNET.2023.332734532:3(1857-1872)Online publication date: Jun-2024
https://doi.org/10.1109/TNET.2023.3327345
Li FChen SJia XGao CWang PWang XCao J(2024)Distributed Program Deployment for Resource-Aware Programmable SwitchesIEEE Transactions on Computers10.1109/TC.2024.335578673:5(1357-1370)Online publication date: 18-Jan-2024
https://dl.acm.org/doi/10.1109/TC.2024.3355786
Kon PGattani ASaharia DCao TBarradas DChen ASherr MUjcich B(2024)NetShuffle: Circumventing Censorship with Shuffle Proxies at the Edge2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00036(3497-3514)Online publication date: 19-May-2024
https://doi.org/10.1109/SP54263.2024.00036
Tanaka KUchida TMatsuda YArikawa YKaji SSakamoto T(2024)Transparent Broadband VPN Gateway: Achieving 0.39 Tbps per Tunnel with Bump-in-the-WireIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621086(2318-2327)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621086
Mazloum AAlSabeh AKfoury ECrichigno J(2024)Security applications in P4: Implementation and lessons learnedComputer Networks10.1016/j.comnet.2024.111011(111011)Online publication date: Dec-2024
https://doi.org/10.1016/j.comnet.2024.111011
Xu KJiang DLi YWang XZhang DXie G(2024)MaPComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2024.110590251:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.comnet.2024.110590
Song CKhooi XJoshi RChoi ILi JChan MSchulzrinne HKohler EMaltz DMisra V(2023)Network Load Balancing with In-network Reordering Support for RDMAProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604849(816-831)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604849
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten