research-article

Page Fault Support for Network Controllers

Authors:

Shachar Raindel,

Muli Ben-Yehuda,

Dan TsafrirAuthors Info & Claims

ASPLOS '17: Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems

Pages 449 - 466

https://doi.org/10.1145/3037697.3037710

Published: 04 April 2017 Publication History

Abstract

Direct network I/O allows network controllers (NICs) to expose multiple instances of themselves, to be used by untrusted software without a trusted intermediary. Direct I/O thus frees researchers from legacy software, fueling studies that innovate in multitenant setups. Such studies, however, overwhelmingly ignore one serious problem: direct memory accesses (DMAs) of NICs disallow page faults, forcing systems to either pin entire address spaces to physical memory and thereby hinder memory utilization, or resort to APIs that pin/unpin memory buffers before/after they are DMAed, which complicates the programming model and hampers performance.

We solve this problem by designing and implementing page fault support for InfiniBand and Ethernet NICs. A main challenge we tackle---unique to NICs---is handling receive DMAs that trigger page faults, leaving the NIC without memory to store the incoming data. We demonstrate that our solution provides all the benefits associated with "regular" virtual memory, notably (1) a simpler programming model that rids users from the need to pin, and (2) the ability to employ all the canonical memory optimizations, such as memory overcommitment and demand-paging based on actual use. We show that, as a result, benchmark performance improves by up to 1.9x.

References

[1]

Brian Aker and Mingqiang Zhuang. Memaslap - load testing and benchmarking a server. http://docs.libmemcached.org/bin/memaslap.html. libmemcached 1.1.0 documentation. Accessed: May 2016.

[2]

M. Alizadeh, B. Atikoglu, A. Kabbani, A. Lakshmikantha, Rong Pan, B. Prabhakar, and M. Seaman. Data center transport mechanisms: Congestion control theory and IEEE standardization. In Communication, Control, and Computing, 2008 46th Annual Allerton Conference on, pages 1270--1277, Sept 2008. http://dx.doi.org/10.1109/ALLERTON.2008.4797706.

[3]

M. Allman, V. Paxson, and W. Stevens. TCP Congestion Control. RFC 2581, Internet Engineering Task Force, April 1999.

Digital Library

[4]

AMD Inc. AMD IOMMU architectural specification, rev 2.00. http://developer.amd.com/wordpress/media/2012/10/488821.pdf, Mar 2011. Accessed: May 2016.

[5]

Nadav Amit, Muli Ben-Yehuda, Dan Tsafrir, and Assaf Schuster. vIOMMU: efficient IOMMU emulation. In USENIX Annual Technical Conference (ATC), pages 73--86, 2011. http://www.usenix.org/events/atc11/tech/final_files/Amit.pdf.

Digital Library

[6]

Nadav Amit, Abel Gordon, Nadav Har'El, Muli Ben-Yehuda, Alex Landau, Assaf Schuster, and Dan Tsafrir. Bare-metal performance for virtual machines with exitless interrupts. Communications of the ACM (CACM), 59(1):108--116, Jan 2016. http://dx.doi.org/10.1145/2845648.

Digital Library

[7]

Nadav Amit, Dan Tsafrir, and Assaf Schuster. VSwapper: A memory swapper for virtualized environments. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 349--366, 2014. http://dx.doi.org/10.1145/2541940.2541969.

Digital Library

[8]

Apple Inc. Thunderbolt device driver programming guide: Debugging VT-d I/O MMU virtualization. https://developer.apple.com/library/mac/documentation/HardwareDrivers/Conceptual/ThunderboltDevGuide/DebuggingThunderboltDrivers/DebuggingThunderboltDrivers.html, 2013. Accessed: May 2014.

[9]

Andrea Arcangeli. Integrating KVM with the linux memory management. In KVM Forum, 2008.

[10]

ARM Holdings. ARM system memory management unit architecture specification -- SMMU architecture version 2.0. http://infocenter.arm.com/help/topic/com.arm.doc.ihi0062c/IHI0062C_system_mmu_architecture_specification.pdf, 2013. Accessed: Jan 2015.

[11]

Jens Axboe. Fio -- flexible IO tester. http://git.kernel.dk/?p=fio.git.

[12]

Thomas Ball, Ella Bounimova, Byron Cook, Vladimir Levin, Jakob Lichtenberg, Con McGarvey, Bohus Ondrusek, Sriram K. Rajamani, and Abdullah Ustuner. Thorough static analysis of device drivers. In ACM Eurosys, pages 73--85, 2006.

Digital Library

[13]

Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. IX: A protected dataplane operating system for high throughput and low latency. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 49--65, 2014. https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-belay.pdf.

[14]

Christian Bell and Dan Bonachea. A new DMA registration strategy for pinning-based high performance networks. In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2003. http://dx.doi.org/10.1109/IPDPS.2003.1213363.

[15]

Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, and Dan Tsafrir. The nom profit-maximizing operating system. In ACM International Conference on Virtual Execution Environments (VEE), pages 145--160, 2016. http://dx.doi.org/10.1145/2892242.2892250.

Digital Library

[16]

Muli Ben-Yehuda, Michael D. Day, Zvi Dubitzky, Michael Factor, Nadav Har'El, Abel Gordon, Anthony Liguori, Orit Wasserman, and Ben-Ami Yassour. The Turtles project: Design and implementation of nested virtualization. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 423--436, 2010. http://www.usenix.org/events/osdi10/tech/full_papers/Ben-Yehuda.pdf.

[17]

Muli Ben-Yehuda, Jimi Xenidis, Michal Ostrowski, Karl Rister, Alexis Bruemmer, and Leendert van Doorn. The price of safety: Evaluating IOMMU performance. In Ottawa Linux Symposium (OLS), pages 9--20, 2007. https://www.kernel.org/doc/mirror/ols2007v1.pdf#page=9.

[18]

Ravi Bhargava, Benjamin Serebrin, Francesco Spadini, and Srilatha Manne. Accelerating two-dimensional page walks for virtualized systems. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 26--35, 2008. http://dx.doi.org/10.1145/1346281.1346286.

Digital Library

[19]

Robert Birke, Lydia Y Chen, and Evgenia Smirni. Data centers in the wild: A large performance study. Technical Report RZ3820, IBM Research, 2012. http://domino.research.ibm.com/library/cyberdig.nsf/papers/0C306B31CF0D3861852579E40045F17F.

[20]

James E.J. Bottomley. Dynamic DMA mapping using the generic device. https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/tree/Documentation/DMA-API.txt?id=refs/tags/v3.18.3. Linux kernel documentation. Accessed: Jan 2015.

[21]

Ethan Burns. Implementation and comparison of iSCSI over RDMA. PhD thesis, University of New Hampshire, 2008.

[22]

Mallikarjun Chadalapaka, Uri Elzur, Michael Ku, Hemal Shah, and Patricia Thaler. A Study of iSCSI Extensions for RDMA. In Computer-Communication Networks, August 2003.

Digital Library

[23]

Yuqun Chen, Angelos Bilas, Stefanos N. Damianakis, Cezary Dubnicki, and Kai Li. UTLB: A mechanism for address translation on network interfaces. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 193--204, 1998. http://dx.doi.org/10.1145/291069.291046.

Digital Library

[24]

Andy Chou, Junfeng Yang, Benjamin Chelf, Seth Hallem, and Dawson Engler. An empirical study of operating systems errors. In ACM Symposium on Operating Systems Principles (SOSP), pages 73--88, 2001.

Digital Library

[25]

Jonathan Corbet. Linux Device Drivers, chapter 15: Memory Mapping and DMA. O'Reilly, 3rd edition, 2005.

[26]

Jonathan Corbet. Newer, newer NAPI. LWN https://lwn.net/Articles/244640/, Aug 2007. (Accessed: Aug 2016).

[27]

Intel Corporation. Intel MPI benchmarks. https://software.intel.com/en-us/articles/intel-mpi-benchmarks, 2013.

[28]

Crehan Research. Another year of robust growth and record shipments for branded data center switches. http://www.crehanresearch.com/wp-content/uploads/2015/03/CREHAN-2014-Data-Center-Switching-CR.pdf, Mar 2015. (Accessed: Aug 2015).

[29]

Yaozu Dong, Yu Chen, Zhenhao Pan, Jinquan Dai, and Yunhong Jiang. ReNIC: Architectural extension to SR-IOV I/O virtualization for efficient replication. ACM Transactions on Architecture and Code Optimization (TACO), 8(4):40:1--40:22, Jan 2012. http://dx.doi.org/10.1145/2086696.2086719.

Digital Library

[30]

Aleksandar Dragojević, Dushyanth Narayanan, Miguel Castro, and Orion Hodson. FaRM: Fast remote memory. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 401--414, 2014. https://www.usenix.org/conference/nsdi14/technical-sessions/dragojevic.

Digital Library

[31]

Adam Dunkels. Design and implementation of the lwIP TCP/IP stack. Swedish Institute of Computer Science, 2:77, 2001.

[32]

Montse Farreras, George Almasi, Calin Cascaval, and Toni Cortes. Scalable RDMA performance in PGAS languages. In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2009. http://dx.doi.org/10.1109/IPDPS.2009.5161025.

Digital Library

[33]

Brad Fitzpatrick. Distributed caching with memcached. Linux Journal, 2004(124):5, Aug 2004. http://dl.acm.org/citation.cfm?id=1012889.1012894.

Digital Library

[34]

Sally Floyd, Dr. K. K. Ramakrishnan, and David L. Black. The Addition of Explicit Congestion Notification (ECN) to IP. RFC 3168, March 2013.

[35]

Tomonori Fujita and Mike Christie. tgt: Framework for Storage Target Drivers. In Proceedings of the Linux Symposium, July 2006.

[36]

GASNet 1.26.0. https://gasnet.lbl.gov/GASNet-1.26.0.tar.gz, October 2015. (Accessed: May 2016).

[37]

Dror Goldenberg, Michael Kagan, Ran Ravid, and Michael S. Tsirkin. Zero copy sockets direct protocol over InfiniBand -- preliminary implementation and performance analysis. In IEEE Symposium on High Performance Interconnects (HOTI), pages 128--137, 2005. http://dx.doi.org/10.1109/CONECT.2005.35.

Digital Library

[38]

Kinshuk Govil, Dan Teodosiu, Yongqiang Huang, and Mendel Rosenblum. Cellular disco: Resource management using virtual clusters on shared-memory multiprocessors. In ACM Symposium on Operating Systems Principles (SOSP), pages 154--169, 1999. http://dx.doi.org/10.1145/319344.319162.

Digital Library

[39]

Diwaker Gupta, Sangmin Lee, Michael Vrable, Stefan Savage, Alex C. Snoeren, George Varghese, Geoffrey M. Voelker, and Amin Vahdat. Difference engine: Harnessing memory redundancy in virtual machines. Communications of the ACM (CACM), pages 85--93, 2010. http://dx.doi.org/10.1145/1831407.1831429.

Digital Library

[40]

James Hamilton. AWS innovation at scale. https://www.youtube.com/watch?t=113&v=JIQETrFC_SQ, Nov 2014. (Accessed: Aug 2015).

[41]

Nadav Har'El, Abel Gordon, Alex Landau, Muli Ben-Yehuda, Avishay Traeger, and Razya Ladelsky. Efficient and scalable paravirtual I/O system. In USENIX Annual Technical Conference (ATC), pages 231--242, 2013. https://www.usenix.org/system/files/conference/atc13/atc13-harel.pdf.

[42]

Jorrit N. Herder, Herbert Bos, Ben Gras, Philip Homburg, and Andrew S. Tanenbaum. Failure resilience for device drivers. In IEEE/IFIP Annual International Conference on Dependable Systems and Networks (DSN), pages 41--50, 2007.

Digital Library

[43]

Gregory D. Hill and Albert H. Chen. High performance network multiplexing with IX

[44]

. Research report, Stanford University, 2015. http://hselin.com/resources/CS344g_ixplusplus_final%20paper.pdf.

[45]

Michael R. Hines, Abel Gordon, Marcio Silva, Dilma Da Silva, Kyung Ryu, and Muli Ben-Yehuda. Applications know best: Performance-driven memory overcommit with Ginkgo. In IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pages 130--137, 2011. http://dx.doi.org/10.1109/CloudCom.2011.27.

Digital Library

[46]

Eric Horschman. Hypervisor memory management done right. http://blogs.vmware.com/virtualreality/2011/02/hypervisor-memory-management-done-right.html, 2011. (Accessed: May 2016).

[47]

The HSA Foundation. http://www.hsafoundation.com/.

[48]

HSA Foundation. HSA-Drivers-Linux-AMD. https://github.com/HSAFoundation/HSA-Drivers-Linux-AMD. (Accessed: May 2016).

[49]

Woomin Hwang, Yangwoo Roh, Youngwoo Park, Ki-Woong Park, and Kyu Ho Park. HyperDealer: Reference pattern aware instant memory balancing for consolidated virtual machines. In IEEE International Conference on Cloud Computing (CLOUD), pages 426--434, 2014. http://dx.doi.org/10.1109/CLOUD.2010.70.

Digital Library

[50]

IBM Corporation. PowerLinux servers -- 64-bit DMA concepts. http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/topic/liabm/liabmconcepts.htm. Accessed: May 2014.

[51]

IBM Corporation. AIX kernel extensions and device support programming concepts. https://publib.boulder.ibm.com/infocenter/aix/v7r1/topic/com.ibm.aix.kernelext/doc/kernextc/kernextc\_pdf.pdf, 2013. Accssed: May 2014.

[52]

IEEE. Specification for 802.3 full duplex operation. IEEE Standard 802.3x http://dx.doi.org, 1997.

[53]

VMware Inc. Configuring VMDirectPath I/O pass-through devices on a VMware ESX or VMware ESXi host. https://kb.vmware.com/kb/1010789. VMware Knowledge Base. Accessed: Aug 2016.

[54]

InfiniBand Trade Association (IBTA). About InfiniBand. http://www.infinibandta.org/content/pages.php?pg=about\_us\_infiniband. (Accessed: May 2016).

[55]

InfiniBand Trade Association (IBTA). About RoCE. http://www.infinibandta.org/content/pages.php?pg=about\_us\_RoCE. (Accessed: May 2016).

[56]

Intel. PCI-SIG SR-IOV primer: An introduction to SR-IOV technology. http://www.intel.com/content/www/us/en/pci-express/pci-sig-sr-iov-primer-sr-iov-technology-paper.html, Jan 2011.

[57]

Intel Corporation. DPDK: Data plane development kit. http://dpdk.org. (Accessed: May 2016).

[58]

Intel Corporation. Intel virtualization technology for directed I/O - architecture specification - specification - Rev.\ 2.2. http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf, Sep 2013. Accessed: Jan 2015.

[59]

Intel Corporation. Intel virtualization technology for directed I/O - architecture specification - Rev.\ 2.3. http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf, Oct 2014.

[60]

Rick A. Jones. Netperf: A network performance benchmark (Revision 2.0). http://www.netperf.org/netperf/training/Netperf.html, 1995. Accessed: August, 2016.

[61]

M. Frans Kaashoek, Dawson R. Engler, Gregory R. Ganger, Hector M. Brice\ no, Russell Hunt, David Mazières, Thomas Pinckney, Robert Grimm, John Jannotti, and Kenneth Mackenzie. Application performance and flexibility on exokernel systems. In ACM Symposium on Operating Systems Principles (SOSP), pages 52--65, 1997.

Digital Library

[62]

Anuj Kalia, Michael Kaminsky, and David G. Andersen. Design guidelines for high performance RDMA systems. In USENIX Annual Technical Conference (ATC), pages 437--450, 2016. https://www.usenix.org/conference/atc16/technical-sessions/presentation/kalia.

[63]

Antoine Kaufmann, SImon Peter, Naveen Kr. Sharma, Thomas Anderson, and Arvind Krishnamurthy. High performance packet processing with FlexNIC. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 67--81, 2016. http://dx.doi.org/10.1145/2872362.2872367.

Digital Library

[64]

Alice E. Koniges, Rolf Rabenseifner, and Karl Solchenbach. Benchmark design for characterization of balanced high-performance architectures. In Proceedings of the 15th International Parallel & Distributed Processing Symposium, IPDPS '01, pages 196--, Washington, DC, USA, 2001. IEEE Computer Society.

[65]

Yossi Kuperman, Eyal Moscovici, Joel Nider, Razya Ladelsky, Abel Gordon, and Dan Tsafrir. Paravirtual remote I/O. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 49--65, 2016. http://dx.doi.org/10.1145/2872362.2872378.

Digital Library

[66]

George Kyriazis. Heterogeneous system architecture: A technical review. Technical report, AMD Inc., Aug 2012. Rev. 1.0 http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/hsa10.pdf (Accessed: May 2016).

[67]

Joshua LeVasseur, Volkmar Uhlig, Jan Stoess, and Stefan Götz. Unmodified device driver reuse and improved system dependability via virtual machines. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 17--30, 2004. https://www.usenix.org/legacy/publications/library/proceedings/osdi04/tech/full_papers/levasseur/levasseur.pdf.

Digital Library

[68]

Sheng Li, Hyeontaek Lim, Victor W. Lee, Jung Ho Ahn, Anuj Kalia, Michael Kaminsky, David G. Andersen, O. Seongil, Sukhan Lee, and Pradeep Dubey. Architecting to achieve a billion requests per second throughput on a single key-value store server platform. In ACM International Symposium on Computer Architecture (ISCA), pages 476--488, 2015. https://doi.org/10.1145/2749469.2750416.

Digital Library

[69]

Hyeontaek Lim, Dongsu Han, David G. Andersen, and Michael Kaminsky. MICA: A holistic approach to fast in-memory key-value storage. In USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 429--444, 2014. https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/lim.

[70]

getrlimit(2) -- Linux man page. http://linux.die.net/man/2/getrlimit. (Accessed: May 2016).

[71]

mlock(2) -- Linux man page. http://linux.die.net/man/2/mlock. (Accessed: May 2016).

[72]

The include/uapi/linux/resource.h header file of Linux 4.5. http://lxr.free-electrons.com/source/include/uapi/linux/resource.h?v=4.5\#L71. (Accessed: May 2016).

[73]

Jiuxing Liu, Dhabaleswar K. Panda, Jiuxing Liu Dhabaleswar K. P, and Mohammad Banikazemi. Evaluating the impact of RDMA on storage I/O over InfiniBand. In SAN-03 Workshop (in conjunction with HPCA), 2004, 2004.

[74]

Moshe Malka, Nadav Amit, Muli Ben-Yehuda, and Dan Tsafrir. rIOMMU: Efficient IOMMU for I/O devices that employ ring buffers. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 355--368, 2015.

Digital Library

[75]

Moshe Malka, Nadav Amit, and Dan Tsafrir. Efficient intra-operating system protection against harmful DMAs. In USENIX Conference on File and Storage Technologies (FAST), pages 29--44, 2015.

Digital Library

[76]

Vinod Mamtani. DMA directions and Windows. http://download.microsoft.com/download/a/f/d/afdfd50d-6eb9--425e-84e1-b4085a80e34e/sys-t304\_wh07.pptx, 2007. Accessed: May 2014.

[77]

Alex Markuze, Adam Morrison, and Dan Tsafrir. It's DAMN time for overhead-free IOMMU protection. Submitted.

[78]

Alex Markuze, Adam Morrison, and Dan Tsafrir. True IOMMU protection from DMA attacks: When copy is faster than zero copy. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 249--262, 2016. http://dx.doi.org/10.1145/2872362.2872379.

Digital Library

[79]

Paul Menage. Cgroups. https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt.

[80]

Frank Mietke, Robert Rex, Robert Baumgartl, Torsten Mehlan, Torsten Hoefler, and Wolfgang Rehm. Analysis of the memory registration process in the Mellanox InfiniBand software stack. In International European Conference on Parallel and Distributed Computing (Euro-Par), pages 124--133, 2006. http://dx.doi.org/10.1007/11823285_13.

Digital Library

[81]

Juan Navarro, Sitaram Iyer, Peter Druschel, and Alan Cox. Practical, transparent operating system support for superpages. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 89--104, 2002.

[82]

Jarek Nieplocha, Vinod Tipparaju, Amina Saify, and Dhabaleswar K. Panda. Protocols and strategies for optimizing performance of remote memory operations on clusters. In IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2002. http://dx.doi.org/10.1109/IPDPS.2002.1016563.

[83]

Radhika Niranjan Mysore, George Porter, and Amin Vahdat. FasTrak: Enabling express lanes in multi-tenant data centers. In ACM Conference on Emerging Networking Experiments and Technologies (CoNEXT), pages 139--150, 2013. http://dx.doi.org/10.1145/2535372.2535386.

Digital Library

[84]

Li Ou, Xubin He, and Jizhong Han. An efficient design for fast memory registration in RDMA. Journal of Network and Computer Applications, 2009.

Digital Library

[85]

Shoumik Palkar, Chang Lan, Sangjin Han, Keon Jang, Aurojit Panda, Sylvia Ratnasamy, Luigi Rizzo, and Scott Shenker. E2: A framework for NFV applications. In ACM Symposium on Operating Systems Principles (SOSP), pages 121--136, 2015. https://doi.org/10.1145/2815400.2815423.

Digital Library

[86]

PCI-SIG. Single root I/O virtualization and sharing 1.0 specification. http://www.pcisig.com/specifications/iov/single_root/, Sep 2007. (Accessed: Aug 2016).

[87]

PCI-SIG. Address Translation Services Revision 1.1. http://www.pcisig.com/specifications/iov/ats/, 2009.

[88]

PCI-SIG. Single root I/O virtualization and sharing 1.1 specification. http://www.pcisig.com/specifications/iov/single_root/, Jan 2010. (Accessed: Aug 2016).

[89]

Omer Peleg, Adam Morrison, Benjamin Serebrin, and Dan Tsafrir. Utilizing the IOMMU Scalably. In USENIX Annual Technical Conference (ATC), 2015.

[90]

Simon Peter, Jialin Li, Doug Woos, Irene Zhang, Dan R. K. Ports, Thomas Anderson, Arvind Krishnamurthy, and Mark Zbikowski. Towards high-performance application-level storage management. In USENIX Workshop on Hot Topics in Storage and File Systems (HOTSTORAGE), 2014. https://www.usenix.org/system/files/conference/hotstorage14/hotstorage14-paper-peter.pdf.

[91]

Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. Arrakis: The operating system is the control plane. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 1--16, 2014. https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-peter_simon.pdf.

Digital Library

[92]

Renato J. Recio, Bernard Metzler, Paul R. Culley, Jeff Hilland, and Dave Garcia. A remote direct memory access protocol specification. RFC 5040, The Internet Engineering Task Force (IETF) Network Working Group, 2007. https://tools.ietf.org/html/rfc5040 (Accessed: May 2016).

[93]

Bruce Richardson. [dpdk-dev] memory pinning. http://dpdk.org/ml/archives/dev/2014-June/003937.html, 2014. (Accessed: Aug 2016).

[94]

Luigi Rizzo. Netmap: A novel framework for fast packet I/O. In USENIX Annual Technical Conference (ATC), pages 101--112, 2012. https://www.usenix.org/conference/atc12/technical-sessions/presentation/rizzo.

[95]

Phil Rogers. Heterogeneous System Architecture (HSA): Overview and implementation. In Hot Chips, 2013. HC25. http://www.hotchips.org/wp-content/uploads/hc_archives/hc25/HC25.0T1-Hetero-epub/HC25.25.100-Intro-Rogers-HSA%20Intro%20HotChips2013_Final.pdf (Accessed: May 2016).

[96]

Arvind Seshadri, Mark Luk, Ning Qu, and Adrian Perrig. SecVisor: A tiny hypervisor to provide lifetime kernel code integrity for commodity OSes. In ACM Symposium on Operating Systems Principles (SOSP), pages 335--350, 2007. http://dx.doi.org/10.1145/1294261.1294294.

Digital Library

[97]

Jiaxin Shi, Youyang Yao, Rong Chen, Haibo Chen, and Feifei Li. Fast and concurrent RDF queries with RDMA-based distributed graph exploration. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 317--332, 2016. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/shi.

[98]

Igor Smolyar, Muli Ben-Yehuda, and Dan Tsafrir. Securing self-virtualizing Ethernet devices. In USENIX Security Symposium, pages 335--350, 2015.

[99]

Livio Soares and Michael Stumm. FlexSC: Flexible system call scheduling with exception-less system calls. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 33--46, 2010. https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Soares.pdf.

Digital Library

[100]

Vaidyanathan Srinivasan, Anand K. Santhanam, and Madhavan Srinivasan. Cell Broadband Engine processor DMA engines, Part 1: The little engines that move data. http://www.ibm.com/developerworks/library/pa-celldmas, 2005. (Accessed: May 2016).

[101]

B. Stephens, A.L. Cox, A. Singla, J. Carter, C. Dixon, and W. Felter. Practical DCB for improved data center networks. In INFOCOM, 2014 Proceedings IEEE, pages 1824--1832, April 2014.

[102]

Michael Swift, Brian Bershad, and Henry Levy. Improving the reliability of commodity operating systems. ACM Transactions on Computer Systems (TOCS), 23(1):77--110, Feb 2005.

Digital Library

[103]

Taneja Group. Hypervisor shootout: Maximizing workload density in the virtualization platform. http://www.vmware.com/files/pdf/vmware-maximize-workload-density-tg.pdf, 2010. (Accessed: May 2016).

[104]

Hiroshi Tezuka, Francis O'Carroll, Atsushi Hori, and Yutaka Ishikawa. Pin-down cache: A virtual memory management technique for zero-copy communication. In IEEE International Parallel Processing Symposium (IPPS), pages 308--314, 1998. http://dx.doi.org/10.1109/IPPS.1998.669932.

[105]

Animesh Trivedi. Remote direct memory access (RDMA) 101 -- quick history lesson and introduction. http://0x8086.blogspot.com/2011/11/remote-direct-memory-access-rdma-101.html, 2011. (Accessed: May 2016).

[106]

Cheng-Chun Tu, Michael Ferdman, Chao-tung Lee, and Tzi-cker Chiueh. A comprehensive implementation and evaluation of direct interrupt delivery. In ACM International Conference on Virtual Execution Environments (VEE), pages 1--15, 2016. http://dx.doi.org/10.1145/2731186.2731189.

Digital Library

[107]

Cheng-Chun Tu, Chao-tang Lee, and Tzi-cker Chiueh. Marlin: A memory-based rack area network. In ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), pages 125--136, 2014. http://doi.acm.org/10.1145/2658260.2658262.

[108]

Gabrie van Zanten. Memory overcommit in production? YES YES YES. http://www.gabesvirtualworld.com/memory-overcommit-in-production-yes-yes-yes/, 2010. (Accessed: May 2016).

[109]

Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. Large-scale cluster management at Google with Borg. In Proceedings of the Tenth European Conference on Computer Systems, EuroSys '15, pages 18:1--18:17, New York, NY, USA, 2015. ACM.

Digital Library

[110]

Carl A. Waldspurger. Memory resource management in VMware ESX server. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 181--194, 2002. https://www.usenix.org/legacy/events/osdi02/tech/waldspurger.html.

[111]

Xingda Wei, Jiaxin Shi, Yanzhe Chen, Rong Chen, and Haibo Chen. Fast in-memory transaction processing using RDMA and H™. In ACM Symposium on Operating Systems Principles (SOSP), pages 87--104, 2015. https://doi.org/10.1145/2815400.2815419.

Digital Library

[112]

Wikipedia. iWARP -- internet Wide Area RDMA Protocol. https://en.wikipedia.org/wiki/IWARP. (Accessed: Aug 2016).

[113]

Dan Williams, Patrick Reynolds, Kevin Walsh, Emin Gün Sirer, and Fred B. Schneider. Device driver safety through a reference validation mechanism. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 241--254, 2008.

Digital Library

[114]

Alex Williamson. VFIO: A user's perspective. In KVM Forum, 2012. http://www.linux-kvm.org/images/b/b4/2012-forum-VFIO.pdf. (Accsessed: May 2016).

[115]

Alex Williamson. [qemu-devel] Intel IOMMU guest emulation and vfio-pci passthrough. https://lists.gnu.org/archive/html/qemu-devel/2015--11/msg04284.html, Nov 2015. (Accessed: Aug 2016).

[116]

Paul Willmann, Scott Rixner, and Alan L. Cox. Protection strategies for direct access to virtualized I/O devices. In USENIX Annual Technical Conference (ATC), pages 15--28, 2008. https://www.usenix.org/legacy/event/usenix08/tech/full_papers/willmann/willmann.pdf.

Digital Library

[117]

Timothy Wood, Gabriel Tarasuk-Levin, Prashant Shenoy, Peter Desnoyers, Emmanuel Cecchet, and Mark D. Corner. Memory buddies: Exploiting page sharing for smart colocation in virtualized data centers. In ACM International Conference on Virtual Execution Environments (VEE), pages 31--40, 2009. http://dx.doi.org/10.1145/1508293.1508299.

Digital Library

[118]

Jiesheng Wu, Pete Wyckoff, and Dhabaleswar Panda. PVFS over InfiniBand: Design and performance evaluation. In International Conference on Parallel Processing (ICPP), pages 125--132, 2003. http://dx.doi.org/10.1109/ICPP.2003.1240573.

[119]

Xiaowei Yang, Chuan Ye, and Qiangmin Lin. Evaluation and enhancement to memory sharing and swapping in Xen 4.1. In Xen Summit, 2011. http://tinyurl.com/xen-mem-share-swap (Accessed: May 2016).

[120]

Ben-Ami Yassour, Muli Ben-Yehuda, and Orit Wasserman. On the DMA mapping problem in direct device assignment. In ACM International Systems and Storage Conference (SYSTOR), pages 18:1--18:12, 2010. http://dx.doi.org/10.1145/1815695.1815718.

Digital Library

Cited By

Yang ZWang QLiao XLu YHuang KShu JMa XWon Y(2024)TeRMProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650698(1-16)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650698
Yang ZWang QLiao XLu YHuang KShu J(2024)Efficiently Enlarging RDMA-Attached Memory with SSDACM Transactions on Storage10.1145/370077221:2(1-27)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3700772
Guo KLi DLuo BShen YPeng KLuo NDai SLiang CSong JYang HZhang XMi ZWitchel EArpaci-Dusseau ARossbach CKeeton K(2024)VPRI: Efficient I/O Page Fault Handling via Software-Hardware Co-Design for IaaS CloudsProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles10.1145/3694715.3695957(541-557)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3694715.3695957
Show More Cited By

Index Terms

Page Fault Support for Network Controllers

Recommendations

Page Fault Support for Network Controllers
Asplos'17

Direct network I/O allows network controllers (NICs) to expose multiple instances of themselves, to be used by untrusted software without a trusted intermediary. Direct I/O thus frees researchers from legacy software, fueling studies that innovate in ...
Page Fault Support for Network Controllers
ASPLOS '17

Direct network I/O allows network controllers (NICs) to expose multiple instances of themselves, to be used by untrusted software without a trusted intermediary. Direct I/O thus frees researchers from legacy software, fueling studies that innovate in ...
Dynamic scratchpad memory management for code in portable systems with an MMU

In this work, we present a dynamic memory allocation technique for a novel, horizontally partitioned memory subsystem targeting contemporary embedded processors with a memory management unit (MMU). We propose to replace the on-chip instruction cache ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASPLOS '17: Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems

April 2017

856 pages

ISBN:9781450344654

DOI:10.1145/3037697

General Chairs:
Yunji Chen
Institute of Computing Technology, CAS, China
,
Olivier Temam
Google, USA
,
Program Chair:
John Carter
IBM, USA

ACM SIGPLAN Notices Volume 52, Issue 4
ASPLOS '17
April 2017
811 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3093336
Editor:
Matthew Fluet
Issue’s Table of Contents
ACM SIGARCH Computer Architecture News Volume 45, Issue 1
Asplos'17
March 2017
812 pages
ISSN:0163-5964
DOI:10.1145/3093337
Editor:
Babak Falsafi
Interim
Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ASPLOS '17

Sponsor:

ASPLOS '17: Architectural Support for Programming Languages and Operating Systems

April 8 - 12, 2017

Xi'an, China

Acceptance Rates

ASPLOS '17 Paper Acceptance Rate 53 of 320 submissions, 17%;

Overall Acceptance Rate 535 of 2,713 submissions, 20%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
987
Total Downloads

Downloads (Last 12 months)160
Downloads (Last 6 weeks)4

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang ZWang QLiao XLu YHuang KShu JMa XWon Y(2024)TeRMProceedings of the 22nd USENIX Conference on File and Storage Technologies10.5555/3650697.3650698(1-16)Online publication date: 27-Feb-2024
https://dl.acm.org/doi/10.5555/3650697.3650698
Yang ZWang QLiao XLu YHuang KShu J(2024)Efficiently Enlarging RDMA-Attached Memory with SSDACM Transactions on Storage10.1145/370077221:2(1-27)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3700772
Guo KLi DLuo BShen YPeng KLuo NDai SLiang CSong JYang HZhang XMi ZWitchel EArpaci-Dusseau ARossbach CKeeton K(2024)VPRI: Efficient I/O Page Fault Handling via Software-Hardware Co-Design for IaaS CloudsProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles10.1145/3694715.3695957(541-557)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3694715.3695957
Yuan YWang RRanganathan NRao NKumar SLantz PSanjeepan VCabrera JKwatra ASankaran RJeong IKim N(2024)Intel Accelerators Ecosystem: An SoC-Oriented Perspective : Industry Product2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA)10.1109/ISCA59077.2024.00066(848-862)Online publication date: 29-Jun-2024
https://doi.org/10.1109/ISCA59077.2024.00066
Li HBerger DHsu LErnst DZardoshti PNovakovic SShah MRajadnya SLee SAgarwal IHill MFontoura MBianchini RAamodt TJerger NSwift M(2023)Pond: CXL-Based Memory Pooling Systems for Cloud PlatformsProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 210.1145/3575693.3578835(574-587)Online publication date: 27-Jan-2023
https://dl.acm.org/doi/10.1145/3575693.3578835
Berger DErnst DLi HZardoshti PShah MRajadnya SLee SHsu LAgarwal IHill MBianchini R(2023)Design Tradeoffs in CXL-Based Memory Pools for Public Cloud PlatformsIEEE Micro10.1109/MM.2023.324158643:2(30-38)Online publication date: 1-Mar-2023
https://doi.org/10.1109/MM.2023.3241586
Tirumalasetty CChou CReddy NGratz PAbouelwafa A(2022)Reducing Minor Page Fault Overheads through Enhanced Page WalkerACM Transactions on Architecture and Code Optimization10.1145/354714219:4(1-26)Online publication date: 16-Sep-2022
https://dl.acm.org/doi/10.1145/3547142
Pismenny BLiss LMorrison ATsafrir DFalsafi BFerdman MLu SWenisch T(2022)The benefits of general-purpose on-NIC memoryProceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems10.1145/3503222.3507711(1130-1147)Online publication date: 28-Feb-2022
https://dl.acm.org/doi/10.1145/3503222.3507711
Psistakis AChrysos NChaix FAsiminakis MGianioudis MXirouchakis PPapaefstathiou VKatevenis M(2022)Optimized Page Fault Handling During RDMAIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2022.317566633:12(3990-4005)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TPDS.2022.3175666
Lv CZhang FGao XZhu C(2022)LA-vIOMMU: An Efficient Hardware-Software Co-design of IOMMU Virtualization2022 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)10.1109/ISPA-BDCloud-SocialCom-SustainCom57177.2022.00038(246-253)Online publication date: Dec-2022
https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom57177.2022.00038
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten