Lukas Werling
Frank Bellosa
ABSTRACT
Persistent memory (PM) provides advantages such as direct access from the CPU at low latency, but it turned out that its performance is sensitive to certain access patterns. Particularly, the access latency increases when the PM is under parallel load. This leads not only to degraded PM performance, but slows down the entire system and wastes energy due to busy, but stalling CPUs. We propose an efficiency metric for measuring the CPU cost of a PM storage system. We evaluate our metric and the power consumption of existing file systems and explore PM-optimized memory copy routines to improve their efficiency. Finally, we propose an alternative system design where access to PM is mediated by an FPGA while allowing selective user space access to files.
- 2021. DMAEngine documentation. https://www.kernel.org/doc/html/v5.15/driver-api/dmaengine/index.html.Google Scholar
- 2022. Intel® Visualization Technology for Directed I/O Architecture Specification. https://www.intel.com/content/www/us/en/content-details/774206/intel-virtualization-technology-for-directed-i-o-architecture-specification.htmlGoogle Scholar
- 2023. Compute Express Link™: The Breakthrough CPU-to-Device Interconnect. https://www.computeexpresslink.org.Google Scholar
- Jens Axboe. 2019. Efficient IO with io_uring. https://kernel.dk/io_uring.pdfGoogle Scholar
- Jens Axboe. 2023. Flexible I/O Tester. https://github.com/axboe/fio.Google Scholar
- Matias Bjørling, Jens Axboe, David Nellans, and Philippe Bonnet. 2013. Linux Block IO: Introducing Multi-Queue SSD Access on Multi-Core Systems. In Proceedings of the 6th International Systems and Storage Conference (Haifa, Israel) (SYSTOR '13). Association for Computing Machinery, New York, NY, USA, Article 22, 10 pages. Google ScholarDigital Library
- Adrian M. Caulfield, Todor I. Mollov, Louis Alex Eisner, Arup De, Joel Coburn, and Steven Swanson. 2012. Providing Safe, User Space Access to Fast, Solid State Disks. In Proceedings of the Seventeenth International Conference on Architectural Support for Programming Languages and Operating Systems (London, England, UK) (ASPLOS XVII). Association for Computing Machinery, New York, NY, USA, 387--400. Google ScholarDigital Library
- Subramanya R. Dulloor, Sanjay Kumar, Anil Keshavamurthy, Philip Lantz, Dheeraj Reddy, Rajesh Sankaran, and Jeff Jackson. 2014. System Software for Persistent Memory. In Proceedings of the Ninth European Conference on Computer Systems (Amsterdam, The Netherlands) (EuroSys '14). Association for Computing Machinery, New York, NY, USA, Article 15, 15 pages. Google ScholarDigital Library
- Bryan Harris and Nihat Altiparmak. 2022. When Poll is More Energy Efficient than Interrupt. In Proceedings of the 14th ACM Workshop on Hot Topics in Storage and File Systems (Virtual Event) (HotStorage '22). Association for Computing Machinery, New York, NY, USA, 59--64. Google ScholarDigital Library
- Tejun Heo and Florian Mickler. 2010. Concurrency Managed Workqueue (cmwq). https://www.kernel.org/doc/html/v5.15/core-api/workqueue.html.Google Scholar
- Intel. 2023. Intel® Data Streaming Accelerator User Guide. https://www.intel.com/content/www/us/en/content-details/759709/intel-data-streaming-accelerator-user-guide.htmlGoogle Scholar
- Joseph Izraelevitz, Jian Yang, Lu Zhang, Juno Kim, Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R. Dulloor, Jishen Zhao, and Steven Swanson. 2019. Basic Performance Measurements of the Intel Optane DC Persistent Memory Module. Google ScholarCross Ref
- Rohan Kadekodi, Saurabh Kadekodi, Soujanya Ponnapalli, Harshad Shirwadkar, Gregory R. Ganger, Aasheesh Kolli, and Vijay Chidambaram. 2021. WineFS: A Hugepage-Aware File System for Persistent Memory That Ages Gracefully. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles (Virtual Event, Germany) (SOSP '21). Association for Computing Machinery, New York, NY, USA, 804--818. Google ScholarDigital Library
- Rohan Kadekodi, Se Kwon Lee, Sanidhya Kashyap, Taesoo Kim, Aasheesh Kolli, and Vijay Chidambaram. 2019. SplitFS: Reducing Software Overhead in File Systems for Persistent Memory. In Proceedings of the 27th ACM Symposium on Operating Systems Principles (Huntsville Ontario Canada, 2019-10-27). ACM, 494--508. Google ScholarDigital Library
- Tamás Koczka. 2023. Learnings from kCTF VRP's 42 Linux kernel exploits submissions. https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.htmlGoogle Scholar
- Youngjin Kwon, Henrique Fingler, Tyler Hunt, Simon Peter, Emmett Witchel, and Thomas Anderson. 2017. Strata: A Cross Media File System. In Proceedings of the 26th Symposium on Operating Systems Principles (Shanghai, China) (SOSP '17). Association for Computing Machinery, New York, NY, USA, 460--477. Google ScholarDigital Library
- Thai Le, Steven Briscoe, and Jonatha Stern. 2017. Fast memcpy with SPDK and Intel® I/OAT DMA Engine. https://www.intel.com/content/www/us/en/developer/articles/technical/fast-memcpy-using-spdk-and-ioat-dma-engine.htmlGoogle Scholar
- Jingbo Su, Jiahao Li, Luofan Chen, Cheng Li, Kai Zhang, Liang Yang, and Yinlong Xu. 2023. Revitalizing the Forgotten On-Chip DMA to Expedite Data Movement in NVM-based Storage Systems. In 21st USENIX Conference on File and Storage Technologies (FAST 23). USENIX Association, Santa Clara, CA, 363--378. https://www.usenix.org/conference/fast23/presentation/suGoogle ScholarDigital Library
- Yan Sun, Yifan Yuan, Zeduo Yu, Reese Kuper, Ipoom Jeong, Ren Wang, and Nam Sung Kim. 2023. Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices. arXiv:cs.PF/2303.15375 Google ScholarCross Ref
- Sidharth Sundar, William Simpson, Jacob Higdon, Caeden Whitaker, Bryan Harris, and Nihat Altiparmak. 2023. Energy Implications of IO Interface Design Choices. In Proceedings of the 15th ACM Workshop on Hot Topics in Storage and File Systems (Boston, MA, USA) (HotStorage '23). Association for Computing Machinery, New York, NY, USA, 58--64. Google ScholarDigital Library
- Zixuan Wang, Xiao Liu, Jian Yang, Theodore Michailidis, Steven Swanson, and Jishen Zhao. 2020. Characterizing and Modeling Non-Volatile Memory Systems. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 496--508. Google ScholarCross Ref
- Lingfeng Xiang, Xingsheng Zhao, Jia Rao, Song Jiang, and Hong Jiang. 2022. Characterizing the Performance of Intel Optane Persistent Memory: A Close Look at Its on-DIMM Buffering. In Proceedings of the Seventeenth European Conference on Computer Systems (New York, NY, USA, 2022-03-28) (EuroSys '22). Association for Computing Machinery, 488--505. Google ScholarDigital Library
- Jian Xu and Steven Swanson. 2016. NOVA: A Log-Structured File System for Hybrid Volatile/Non-Volatile Main Memories. In Proceedings of the 14th Usenix Conference on File and Storage Technologies (FAST'16). 323--338. https://www.usenix.org/conference/fast16/technical-sessions/presentation/xuGoogle ScholarDigital Library
- Jian Yang, Juno Kim, Morteza Hoseinzadeh, Joseph Izraelevitz, and Steve Swanson. 2020. An Empirical Guide to the Behavior and Use of Scalable Persistent Memory. In 18th USENIX Conference on File and Storage Technologies (FAST 20). 169--182. https://www.usenix.org/conference/fast20/presentation/yangGoogle ScholarDigital Library
- Yang Yang, Qiang Cao, Jie Yao, Yuanyuan Dong, and Weikang Kong. 2021. SPMFS: A Scalable Persistent Memory File System on Optane Persistent Memory. In Proceedings of the 50th International Conference on Parallel Processing (Lemont, IL, USA) (ICPP '21). Association for Computing Machinery, New York, NY, USA, Article 3, 10 pages. Google ScholarDigital Library
- Shengan Zheng, Morteza Hoseinzadeh, and Steven Swanson. 2019. Ziggurat: A Tiered File System for Non-Volatile Main Memories and Disks. In 17th USENIX Conference on File and Storage Technologies (FAST 19). 207--219. https://www.usenix.org/conference/fast19/presentation/zhengGoogle Scholar
- Diyu Zhou, Yuchen Qian, Vishal Gupta, Zhifei Yang, Changwoo Min, and Sanidhya Kashyap. 2022. ODINFS: Scaling PM Performance with Opportunistic Delegation. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). 179--193. https://www.usenix.org/conference/osdi22/presentation/zhou-diyuGoogle Scholar
Recommendations
VM aware journaling: improving journaling file system performance in virtualization environments
Journaling file systems, which are widely used in modern operating systems, guarantee file system consistency and data integrity by logging file system updates to a journal, which is a reserved space on the storage, before the updates are written to the ...
Analyzing the impact of CPU pinning and partial CPU loads on performance and energy efficiency
CCGRID '15: Proceedings of the 15th IEEE/ACM International Symposium on Cluster, Cloud, and Grid ComputingWhile workload colocation is a necessity to increase energy efficiency of contemporary multi-core hardware, it also increases the risk of performance anomalies due to workload interference. Pinning certain workloads to a subset of CPUs is a simple ...
Improving Resource Efficiency at Scale with Heracles
User-facing, latency-sensitive services, such as websearch, underutilize their computing resources during daily periods of low traffic. Reusing those resources for other tasks is rarely done in production services since the contention for shared ...
Comments