ABSTRACT
DMA operations can access memory buffers only if they are "mapped" in the IOMMU, so operating systems protect themselves against malicious/errant network DMAs by mapping and unmapping each packet immediately before/after it is DMAed. This approach was recently found to be riskier and less performant than keeping packets non-DMAable and instead copying their content to/from permanently-mapped buffers. Still, the extra copy hampers performance of multi-gigabit networking. We observe that achieving protection at the DMA (un)map boundary is needlessly constraining, as devices must be prevented from changing the data only after the kernel reads it. So there is no real need to switch ownership of buffers between kernel and device at the DMA (un)mapping layer, as opposed to the approach taken by all existing IOMMU protection schemes. We thus eliminate the extra copy by (1)~implementing a new allocator called DMA-Aware Malloc for Networking (DAMN), which (de)allocates packet buffers from a memory pool permanently mapped in the IOMMU; (2)~modifying the network stack to use this allocator; and (3)~copying packet data only when the kernel needs it, which usually morphs the aforementioned extra copy into the kernel's standard copy operation performed at the user-kernel boundary. DAMN thus provides full IOMMU protection with performance comparable to that of an unprotected system.
- Brian Aker. Memslap - load testing and benchmarking a server. http://docs.libmemcached.org/bin/memslap.html. libmemcached 1.1.0 documentation. Accessed: Jan 2018.Google Scholar
- AMD Inc. AMD IOMMU architectural specification, rev 2.00. http://developer.amd.com/wordpress/media/2012/10/488821.pdf, Mar 2011. Accessed: Jan 2018.Google Scholar
- Apple Inc. Thunderbolt device driver programming guide: Debugging VT-d I/O MMU virtualization. https://developer.apple.com/library/mac/documentation/HardwareDrivers/Conceptual/ThunderboltDevGuide/DebuggingThunderboltDrivers/DebuggingThunderboltDrivers.html, 2013. Accessed: Jan 2018.Google Scholar
- ARM Holdings. ARM system memory management unit architecture specification -- SMMU architecture version 2.0. http://infocenter.arm.com/help/topic/com.arm.doc.ihi0062d.c/IHI0062D_c_system_mmu_architecture_specification.pdf, 2013. Accessed: Jan 2018.Google Scholar
- Damien Aumaitre and Christophe Devine. Subverting Windows 7 x64 kernel with DMA attacks. In Hack In The Box Security Conference (HITB), 2010. http://esec-lab.sogeti.com/static/publications/10-hitbamsterdam-dmaattacks.pdf. Accessed: Jan 2018.Google Scholar
- Jens Axboe. Flexible I/O Tester. https://github.com/axboe/fio. Accessed: Jan 2018.Google Scholar
- Michael Becher, Maximillian Dornseif, and Christian N. Klein. FireWire: all your memory are belong to us. In CanSecWest Applied Security Conference, 2005. https://cansecwest.com/core05/2005-firewire-cansecwest.pdf. Accessed: Jan 2018.Google Scholar
- Jeff Bonwick and Jonathan Adams. Magazines and Vmem: Extending the Slab allocator to many CPUs and arbitrary resources. In USENIX Annual Technical Conference (ATC), pages 15--44, 2001. https://www.usenix.org/legacy/publications/library/proceedings/usenix01/full_papers/bonwick/bonwick.pdf. Google ScholarDigital Library
- James E.J. Bottomley. Dynamic DMA mapping using the generic device. https://www.kernel.org/doc/Documentation/DMA-API.txt. Linux kernel documentation. Accessed: Jan 2018.Google Scholar
- James E.J. Bottomley. Integrating DMA into the generic device mode. In Ottawa Linux Symposium (OLS), pages 63--75, 2003. https://www.kernel.org/doc/ols/2003/ols2003-pages-63--75.pdf. Accessed: Jan 2018.Google Scholar
- Jonathan Brossard. Hardware backdooring is pratical. In Black Hat, 2012. http://www.toucan-system.com/research/blackhat2012_brossard_hardware_backdooring.pdf. Accessed: Jan 2018.Google Scholar
- Jonathan Corbet. Smarter shrinkers. https://lwn.net/Articles/550463/, May 2013. Accessed: Jan 2018.Google Scholar
- Jonathan Corbet. An introduction to compound pages. https://lwn.net/Articles/619514/, Nov 2014. Accessed: Jan 2018.Google Scholar
- Maximillian Dornseif. 0wned by an iPod. In PACific SECurity -- applied security conferences and training in Pacific Asia (PacSec), 2004. https://pacsec.jp/psj04/psj04-dornseif-e.ppt. Accessed: Jan 2018.Google Scholar
- DPDK. http://dpdk.org/. Accessed: Jan 2018.Google Scholar
- Lo"ıc Duflot, Yves-Alexis Perez, Guillaume Valadon, and Olivier Levillain. Can you still trust your network card? Technical report, French Network and Information Security Agency (FNISA), Mar 2010. http://www.ssi.gouv.fr/uploads/IMG/pdf/csw-trustnetworkcard.pdf. Accessed: Jan 2018.Google Scholar
- Brad Fitzpatrick. Distributed caching with memcached. Linux Journal, 2004(124), Aug 2004. http://www.linuxjournal.com/article/7451. Accessed: Jan 2018. Google ScholarDigital Library
- Google LLC. Google infrastructure security design overview. https://cloud.google.com/security/security-design, Jan 2017. Google Cloud Whitepaper. Accessed: Jan 2018.Google Scholar
- IBM Corporation. PowerLinux servers -- 64-bit DMA concepts. http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/topic/liabm/liabmconcepts.htm. Accessed: Jan 2018.Google Scholar
- IBM Corporation. AIX kernel extensions and device support programming concepts. http://public.dhe.ibm.com/systems/power/docs/aix/71/kernextc_pdf.pdf, 2013. Accssed: Jan 2018.Google Scholar
- Intel TXT Overview. https://www.kernel.org/doc/Documentation/intel_txt.txt. Linux kernel documentation. Accessed: Jan 2018.Google Scholar
- Intel Corporation. Intel trusted execution technology. http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/trusted-execution-technology-security-paper.pdf, 2012. Accessed: Jan 2018.Google Scholar
- Intel Corporation. Intel virtualization technology for directed I/O, architecture specification - architecture specification - Rev. 2.5. http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf, Nov 2017. Accessed: Jan 2018.Google Scholar
- Joerg Roedel. AMD IOMMU DMA-API scalability improvements, Linux patch. https://lists.linuxfoundation.org/pipermail/iommu/2015-December/015245.html, Dec 2015. Accessed: Jan 2018.Google Scholar
- Intel-IOMMU.txt -- Linux IOMMU support. https://www.kernel.org/doc/Documentation/Intel-IOMMU.txt. Linux kernel documentation. Accessed: Jan 2018.Google Scholar
- Moshe Malka, Nadav Amit, Muli Ben-Yehuda, and Dan Tsafrir. rIOMMU: Efficient IOMMU for I/O devices that employ ring buffers. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 355--368, 2015. Google ScholarDigital Library
- Moshe Malka, Nadav Amit, and Dan Tsafrir. Efficient intra-operating system protection against harmful DMAs. In USENIX Conference on File and Storage Technologies (FAST), pages 29--44, 2015. https://www.usenix.org/system/files/conference/fast15/fast15-paper-malka.pdf. Google ScholarDigital Library
- Vinod Mamtani. DMA directions and Windows. http://download.microsoft.com/download/a/f/d/afdfd50d-6eb9--425e-84e1-b4085a80e34e/sys-t304_wh07.pptx, 2007. Accessed: Jan 2018.Google Scholar
- Alex Markuze, Adam Morrison, and Dan Tsafrir. True IOMMU protection from DMA attacks: When copy is faster than zero copy. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 249--262, 2016. Google ScholarDigital Library
- Mellanox Technologies. ConnectX-5 Ex 100,Gb/s Ethernet Single and Dual QSFP28 Port Adapter Cards User Manual. http://www.mellanox.com/related-docs/user_manuals/ConnectX-5_Ethernet_Single_and_Dual_QSFP28_Port_Adapter_Card_User_Manual.pdf, 2018. Accessed: Jan 2018.Google Scholar
- Bosko Milekic. Network buffer allocation in the FreeBSD operating system. In The Technical BSD Conference (BSDCan), 2004. https://www.bsdcan.org/2004/papers/NetworkBufferAllocation.pdf. Accessed: Jan 2018.Google Scholar
- The netfilter.org project. http://www.netfilter.org/. Accessed: Jan 2018.Google Scholar
- Netperf -- a network performance benchmark. https://github.com/HewlettPackard/netperf. Accessed: Jan 2018.Google Scholar
- Omer Peleg, Adam Morrison, Benjamin Serebrin, and Dan Tsafrir. Utilizing the IOMMU Scalably. In USENIX Annual Technical Conference (ATC), pages 549--562, 2015. https://www.usenix.org/system/files/conference/atc15/atc15-paper-peleg.pdf. Google ScholarDigital Library
- Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. Arrakis: The operating system is the control plane. In USENIX Symposium on Operating System Design and Implementation (OSDI), pages 1--16, 2014. https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-peter_simon.pdf. Google ScholarDigital Library
- Arjun Singh, Joon Ong, Amit Agarwal, Glen Anderson, Ashby Armistead, Roy Bannon, Seb Boving, Gaurav Desai, Bob Felderman, Paulie Germano, Anand Kanagala, Hong Liu, Jeff Provost, Jason Simmons, Eiichi Tanda, Jim Wanderer, Urs Hölzle, Stephen Stuart, and Amin Vahdat. Jupiter rising: A decade of Clos topologies and centralized control in Google's datacenter network. Communications of the ACM (CACM), 59(9):88--97, Aug 2016. Google ScholarDigital Library
- SPIEGEL Staff. Inside TAO: Documents Reveal Top NSA Hacking Unit. Der Spiegel, Dec 2013. http://www.spiegel.de/international/world/the-nsa-uses-powerful-toolbox-in-effort-to-spy-on-global-networks-a-940969.html. Accessed: Jan 2018.Google Scholar
- Patrick Stewin and Iurii Bystrov. Understanding DMA malware. In Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA), pages 21--41, 2012. Google ScholarDigital Library
- Arrigo Triulzi. I 0wn the NIC, now I want a shell! In PACific SECurity -- applied security conferences and training in Pacific Asia (PacSec), 2008. http://www.alchemistowl.org/arrigo/Papers/Arrigo-Triulzi-PACSEC08-Project-Maux-II.pdf. Accessed: Jan 2018.Google Scholar
- Thomas Willhalm, Roman Dementiev, and Patrick Fay. Intel performance counter monitor. http://www.intel.com/software/pcm, Jan 2017. Intel Developer Zone. Accessed: Jan 2018.Google Scholar
- Mitch Williams. i40e: enable packet split only when IOMMU present, Linux commit. https://github.com/torvalds/linux/commit/2bc7ee8ac5439efec66fa20a8dc01c0a2b5af739. Accessed: Aug 2018.Google Scholar
- Zongwei Zhou, Virgil D. Gligor, James Newsome, and Jonathan M. McCune. Building verifiable trusted path on commodity x86 computers. In IEEE Symposium on Security and Privacy (S&P), pages 616--630, 2012. Google ScholarDigital Library
Index Terms
- DAMN: Overhead-Free IOMMU Protection for Networking
Recommendations
True IOMMU Protection from DMA Attacks: When Copy is Faster than Zero Copy
ASPLOS '16: Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating SystemsMalicious I/O devices might compromise the OS using DMAs. The OS therefore utilizes the IOMMU to map and unmap every target buffer right before and after its DMA is processed, thereby restricting DMAs to their designated locations. This usage model, ...
DAMN: Overhead-Free IOMMU Protection for Networking
ASPLOS '18DMA operations can access memory buffers only if they are "mapped" in the IOMMU, so operating systems protect themselves against malicious/errant network DMAs by mapping and unmapping each packet immediately before/after it is DMAed. This approach was ...
True IOMMU Protection from DMA Attacks: When Copy is Faster than Zero Copy
ASPLOS'16Malicious I/O devices might compromise the OS using DMAs. The OS therefore utilizes the IOMMU to map and unmap every target buffer right before and after its DMA is processed, thereby restricting DMAs to their designated locations. This usage model, ...
Comments