Abstract
Non-volatile memory, also called persistent memory (PM), has the features of byte addressing, non-volatility and the similar performance with traditional DRAM, but still shows obvious latency in several common scenarios which adopt the synchronous (sync) I/O, such as the application transferring large PM data or accessing the remote PM data in a NUMA architecture. These problems motivate the asynchronous (async) I/O of a PM file system. In this paper, we first investigate the efficiency of the combination of PM and IO_uring, which is a novel and highly-efficient async I/O system proposed recently. We find IO_uring on PM still incurs a serial of performance issues: (1) pseudo-async I/O path; (2) low efficiency memory allocation of I/O data and (3) unnecessary CPU overhead on user polling. Then we introduce LWAIO, a lightweight async I/O system to relieve the above issues. It mainly contains three techniques: (1) kernel-level threading; (2) dynamic memory pool and (3) kernel pushing. We implement LWAIO in NOVA, a well-known PM file system and conduct extensive experiments to verify its advantages on a real PM platform. The experimental results show that LWAIO brings up to 13% IOPS benefit when dealing with random write I/O operation, as well as 45% IOPS improvement when dealing with the random reads.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hu, L., Liu, A., Xie, M., Wang, T.: UAVs joint vehicles as data mules for fast codes dissemination for edge networking in smart city. Peer-to-Peer Netw. Appl. (2019). https://doi.org/10.1007/s12083-019-00752-0
Vučinić, D., et al.: DC express: shortest latency protocol for reading phase change memory over PCI express. In: 12th USENIX Conference on File and Storage Technologies (FAST 14), pp. 309–315. USENIX Association (2014)
Wang, T., Wang, P., Cai, S., Ma, Y., Liu, A., Xie, M.: A unified trustworthy environment establishment based on edge computing in industrial IoT. IEEE Trans. Ind. Inf. 16(9), 6083–6091 (2020). https://doi.org/10.1109/TII.2019.2955152
Li, D., Zhang, N., Dong, M., Chen, H., Ota, K., Tang, Y.: PM-AIO: an effective asynchronous i/o system for persistent memory. IEEE Trans. Emerg. Topics Comput., 1 (2021). https://doi.org/10.1109/TETC.2021.3109047
Chen, Z., Li, D., Wang, Z., Liu, H., Tang, Y.: RAMCI: a novel asynchronous memory copying mechanism based on I/OAT. CCF Trans. High Perf. Comput. 3(2), 129–143 (2021). https://doi.org/10.1007/s42514-021-00063-y
Jens, A.: Efficient io with io_uring (2019). https://kernel.dk/io_uring.pdf
Dulloor, S.R., et al.: System software for persistent memory. In: Proceedings of the Ninth European Conference on Computer Systems, EuroSys ’14. Association for Computing Machinery (2014). https://doi.org/10.1145/2592798.2592814
Corbet, J.: Dax, mmap(), and a “go faster” flag (2016). https://lwn.net/Articles/684828/
Xu, J., Swanson, S.: NOVA: a log-structured file system for hybrid volatile/non-volatile main memories. In: 14th USENIX Conference on File and Storage Technologies (FAST 16), pp. 323–338. USENIX Association, Santa Clara (2016)
Huang, M., Liu, A., Wang, T., Huang, C.: Green data gathering under delay differentiated services constraint for internet of things. Wirel. Commun. Mobi. Comput. (2018). https://doi.org/10.1155/2018/9715428
Kwon, Y., Fingler, H., Hunt, T., Peter, S., Witchel, E., Anderson, T.: Strata: a cross media file system. In: Proceedings of the 26th Symposium on Operating Systems Principles, SOSP ’17, pp. 460–477. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3132747.3132770
Kadekodi, R., Lee, S.K., Kashyap, S., Kim, T., Kolli, A., Chidambaram, V.: Splitfs: reducing software overhead in file systems for persistent memory. In: Proceedings of the 27th ACM Symposium on Operating Systems Principles, SOSP ’19, pp. 494–508. Association for Computing Machinery (2019). https://doi.org/10.1145/3341301.3359631
Vishal, V., John, K.: Improved storage performance using the new linux kernel i/o interface (2019). https://www.snia.org/educational-library/improved-storage-performance-using-new-linux-kernel-io-interface-2019
Izraelevitz, J., et al.: Basic performance measurements of the intel optane dc persistent memory module (2019). https://arxiv.org/abs/1903.05714
Yang, J., Kim, J., Hoseinzadeh, M., Izraelevitz, J., Swanson, S.: An empirical guide to the behavior and use of scalable persistent memory. In: 18th USENIX Conference on File and Storage Technologies (FAST 20), pp. 169–182. USENIX Association, Santa Clara (2020)
Volos, H., Magalhaes, G., Cherkasova, L., Li, J.: Quartz: a lightweight performance emulator for persistent memory software. In: Proceedings of the 16th Annual Middleware Conference, Middleware ’15, pp. 37–49. Association for Computing Machinery (2015)
Lee, G., Shin, S., Song, W., Ham, T.J., Lee, J.W., Jeong, J.: Asynchronous i/o stack: A low-latency kernel i/o stack for ultra-low latency SSDs. In: 2019 USENIX Annual Technical Conference (USENIX ATC 19), Renton, WA, pp. 603–616 (2019)
Acknowledgment
This work was funded by the Key-Area Research and Development Program of Guangdong Province under grant number 2020B0101650001, by the National Natural Science Foundation of China under grant number 61972164, by the Guangdong Basic and Applied Basic Research Foundation under grant number 2019A1515011160, by the Guangzhou Key Laboratory of Big Data and Intelligent Education under grant number 201905010009.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Luo, J., Zhang, W., Li, D., Luo, H., Zeng, D. (2022). A Lightweight Asynchronous I/O System for Non-volatile Memory. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13156. Springer, Cham. https://doi.org/10.1007/978-3-030-95388-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-95388-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95387-4
Online ISBN: 978-3-030-95388-1
eBook Packages: Computer ScienceComputer Science (R0)