Theo Jepsen
Masoud Moshref
Nate Foster
ABSTRACT
In this paper, we explore how programmable data planes can provide a higher-level of service to user applications via a new abstraction called packet subscriptions. Packet subscriptions generalize forwarding rules, and can be used to express both traditional routing and more esoteric, content-based approaches. We describe a compiler for packet subscriptions that uses a novel BDD-based algorithm to efficiently translate predicates into P4 tables that can support O(100K) expressions. Using our compiler, we've built a proof-of-concept pub/sub financial application for splitting market feeds (e.g., Nasdaq's ITCH protocol) with line-rate message processing, using the full switch bandwidth of 6.5Tbps.
Supplemental Material
- S. B. Akers. Binary decision diagrams. IEEE Transactions on Computers (TC), 27(6):509--516, June 1978. Google Scholar
Digital Library
- C. J. Anderson, N. Foster, A. Guha, J.-B. Jeannin, D. Kozen, C. Schlesinger, and D. Walker. Netkat: Semantic foundations for networks. In Symposium on Principles of Programming Languages (POPL), pages 113--126, Jan. 2014. Google ScholarDigital Library
- Apache activemq. http://activemq.apache.org/.Google Scholar
- R. Beckett, A. Gupta, R. Mahajan, and D. Walker. A general approach to network configuration verification. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 155--168, Aug. 2017. Google ScholarDigital Library
- P. Bosshart, D. Daly, G. Gibb, M. Izzard, N. McKeown, J. Rexford, C. Schlesinger, D. Talayco, A. Vahdat, G. Varghese, and D. Walker. P4: Programming Protocol-Independent Packet Processors. SIGCOMM Computer Communication Review (CCR), 44(3):87--95, July 2014. Google ScholarDigital Library
- P. Bosshart, G. Gibb, H.-S. Kim, G. Varghese, N. McKeown, M. Izzard, F. Mujica, and M. Horowitz. Forwarding Metamorphosis: Fast Programmable Match-Action Processing in Hardware for SDN. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 99--110, Aug. 2013. Google ScholarDigital Library
- R. E. Bryant. Graph-based algorithms for boolean function manipulation. IEEE Transactions on Computers (TC), 35(8):677--691, Aug. 1986. Google ScholarDigital Library
- A. Carzaniga and A. L. Wolf. Content-based networking: A new communication infrastructure. In NSF Workshop Infrastructure for Mobile and Wireless Systems (IMWS), pages 59--68, Oct. 2001. Google ScholarDigital Library
- A. Carzaniga and A. L. Wolf. Forwarding in a content-based network. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 163--174, Aug. 2003. Google ScholarDigital Library
- W. Chan, R. Anderson, P. Beame, and D. Notkin. Combining constraint solving and symbolic model checking for a class of systems with nonlinear constraints. In International Conference on Computer Aided Verification (CAV), pages 316--327, June 1997. Google ScholarDigital Library
- D. Clark. The design philosophy of the DARPA Internet Protocols. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 106--114, Aug. 1988. Google ScholarDigital Library
- DPDK. http://dpdk.org/.Google Scholar
- D. E. Eisenbud, C. Yi, C. Contavalli, C. Smith, R. Kononov, E. Mann-Hielscher, A. Cilingiroglu, B. Cheyney, W. Shang, and J. D. Hosein. Maglev: A fast and reliable software network load balancer. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 523--535, Mar. 2016. Google ScholarDigital Library
- P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M. Kermarrec. The many faces of publish/subscribe. ACM Computing Surveys (CSUR), 35(2):114--131, June 2003. Google ScholarDigital Library
- D. Fisher. A look behind the future internet architectures efforts. SIGCOMM Computer Communication Review (CCR), 44(3):45--49, July 2014. Google ScholarDigital Library
- N. Foster, R. Harrison, M. J. Freedman, C. Monsanto, J. Rexford, A. Story, and D. Walker. Frenetic: A Network Programming Language. In International Conference on Functional Programming (ICFP), pages 279--291, Sept. 2011. Google ScholarDigital Library
- IBM MQ. https://www-03.ibm.com/software/products/en/ibm-mq.Google Scholar
- V. Jacobson, D. K. Smetters, J. D. Thornton, M. F. Plass, N. H. Briggs, and R. L. Braynard. Networking named content. In ACM International Conference on Emerging Networking Experiments and Technologies (CoNEXT), pages 1--12, Dec. 2009. Google ScholarDigital Library
- X. Jin, J. Gossels, J. Rexford, and D. Walker. Covisor: A compositional hypervisor for software-defined networks. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 87--101, Oakland, CA, May 2015. Google ScholarDigital Library
- X. Jin, X. Li, H. Zhang, R. Soulé, J. Lee, N. Foster, C. Kim, and I. Stoica. NetCache: Balancing Key-Value Stores with Fast In-Network Caching. In ACM Symposium on Operating Systems Principles (SOSP), Oct. 2017. Google ScholarDigital Library
- Open-sourcing Katran, a scalable network load balancer. https://code.fb.com/open-source/open-sourcing-katran-a-scalable-network-load-balancer/.Google Scholar
- T. Koponen, M. Chawla, B.-G. Chun, A. Ermolinskiy, K. H. Kim, S. Shenker, and I. Stoica. A Data-oriented (and Beyond) Network Architecture. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 181--192, Aug. 2007. Google ScholarDigital Library
- J. Kreps, N. Narkhede, and J. Rao. Kafka: A distributed messaging system for log processing. In 6th International Workshop on Networking Meets Databases (NetDB), June 2011.Google Scholar
- P. Lapukhov. Internet-scale virtual networking using identifier-locator addressing. https://www.nanog.org/sites/default/files/20161018_Lapukhov_Internet-Scale_Virtual_Networking_v1.pdf.Google Scholar
- X. Li, R. Sethi, M. Kaminsky, D. G. Andersen, and M. J. Freedman. Be fast, cheap and in control with switchkv. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 31--44, Mar. 2016. Google ScholarDigital Library
- J. Liu, W. Hallahan, C. Schlesinger, M. Sharif, J. Lee, R. Soulé, H. Wang, C. Caşcaval, N. McKeown, and N. Foster. P4v: Practical verification for programmable data planes. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 490--503, Aug. 2018. Google ScholarDigital Library
- C. Monsanto et al. Composing Software-Defined Networks. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 1--13, Apr. 2013. Google ScholarDigital Library
- S. Narayana, A. Sivaraman, V. Nathan, P. Goyal, V. Arun, M. Alizadeh, V. Jeyakumar, and C. Kim. Language-directed hardware design for network performance monitoring. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 85--98, Aug. 2017. Google ScholarDigital Library
- E. Nordström, D. Shue, P. Gopalan, R. Kiefer, M. Arye, S. Y. Ko, J. Rexford, and M. J. Freedman. Serval: An end-host stack for service-centric networking. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), Apr. 2012. Google ScholarDigital Library
- M. Papalini, A. Carzaniga, K. Khazaei, and A. L. Wolf. Scalable routing for tag-based information-centric networking. In Proceedings of the 1st International Conference on Information-centric Networking (HotICN), pages 17--26, Sept. 2014. Google ScholarDigital Library
- M. Papalini, K. Khazaei, A. Carzaniga, and D. Rogora. High throughput forwarding for ICN with descriptors and locators. In ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 43--54, 2016. Google ScholarDigital Library
- D. Perino, M. Varvello, L. Linguaglossa, R. Laufer, and R. Boislaigue. Caesar: A content router for high-speed forwarding on content names. In ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 137--148, Oct. 2014. Google ScholarDigital Library
- R. Potharaju and N. Jain. Demystifying the dark side of the middle: A field study of middlebox failures in datacenters. In ACM SIGCOMM Internet Measurement Conference (IMC), pages 9--22, Oct. 2013. Google ScholarDigital Library
- J. Sherry, S. Hasan, C. Scott, A. Krishnamurthy, S. Ratnasamy, and V. Sekar. Making middleboxes someone else's problem: Network processing as a cloud service. In Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM), pages 13--24, Aug. 2012. Google ScholarDigital Library
- Siena Synthetic Benchmark Generator. http://www.inf.usi.ch/carzaniga/cbn/forwarding/.Google Scholar
- S. Smolka, S. Eliopoulos, N. Foster, and A. Guha. A fast compiler for netkat. In International Conference on Functional Programming (ICFP), pages 328--341, Sept. 2015. Google ScholarDigital Library
- R. Soulé, S. Basu, P. J. Marandi, F. Pedone, R. Kleinberg, E. G. Sirer, and N. Foster. Merlin: A Language for Provisioning Network Resources. In ACM International Conference on Emerging Networking Experiments and Technologies (CoNEXT), Dec. 2014. Google ScholarDigital Library
- Tibco rendezvous. https://www.tibco.com/products/tibco-rendezvous.Google Scholar
- Y. Wang, B. Xu, D. Tai, J. Lu, T. Zhang, H. Dai, B. Zhang, and B. Liu. Fast name lookup for named data networking. In IEEE International Symposium of Quality of Service (IWQoS), pages 198--207, May 2014.Google ScholarCross Ref
- XPliant Ethernet Switch Product Family. www.cavium.com/XPliant-Ethernet-Switch-Product-Family.html.Google Scholar
- H. Yang and S. S. Lam. Real-time verification of network properties using atomic predicates. In IEEE International Conference on Network Protocols (ICNP), pages 1--11, Oct. 2013.Google ScholarCross Ref
- H. Yuan and P. Crowley. Reliably scalable name prefix lookup. In ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 111--121, May 2015. Google ScholarDigital Library
Recommendations
Reprogrammable network packet processing on the field programmable port extender (FPX)
FPGA '01: Proceedings of the 2001 ACM/SIGDA ninth international symposium on Field programmable gate arraysA prototype platform has been developed that allows processing of packets at the edge of a multi-gigabit-per-second network switch. This system, the Field Programmable Port Extender (FPX), enables packet processing functions to be implemented as modular ...
Forwarding and Routing With Packet Subscriptions
In this paper, we explore how programmable data planes can naturally provide a higher-level of service to user applications via a new abstraction called packet subscriptions. Packet subscriptions generalize forwarding rules, and can be used to express ...
Study of temporal behaviour of packet loss in packet switches with bursty traffic arrivals
The study of packet loss is of great importance to the design of fast packet switching systems. Fast packet switching is generally accepted as the best technique for designing high-speed computer networks. Due to the high throughput demands and the ...
Comments