Skip to main content
SpringerLink
Log in

HasFS: optimizing file system consistency mechanism on NVM-based hybrid storage architecture

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In order to protect the data during system crash, traditional DRAM–DISK architecture file systems (e.g., EXT4) need to synchronize the dirty metadata and data from the memory to disk. At the same time, the disk synchronization may break the consistency of file system upon a crash, so traditional file systems use some mechanisms to guarantee the file system consistency when the dirty metadata and data is synchronized onto persistent storage devices (e.g., HDD and SSD). Journaling is a consistency mechanism widely used by file systems. We observe that the overhead of periodic disk synchronization and journaling is high. Emerging non-volatile memories (NVMs) can be potentially utilized to reduce these overheads. In this paper, we present hybrid architecture for storage file system (HasFS), a file system designed for the DRAM–NVM–DISK architecture. HasFS extends the main memory with NVM and considers NVM as a persistent page cache to eliminate the periodic disk synchronization overhead of dirty data. Then we design an efficient consistency mechanism based on the hybrid memory architecture to provide strong (both metadata and data) consistency guarantee with low overhead. The evaluation demonstrates that HasFS outperforms mainstream DRAM–DISK file systems for many workloads. For instance, HasFS has between 1.6X to 46.6X performance improvement over other tested file systems in random write workload. In particular, HasFS outperforms EXT4 without journal in some cases even though HasFS provides metadata and data consistency guarantees (similar to EXT4 with journal data mode).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. beegfs.io: Beegfs. https://www.beegfs.io (2019)

  2. Bonwick, J., Moore, B.: Zfs: The last word in file systems (2007)

  3. btrfs.org: Btrfs. https://btrfs.wiki.kernel.org (2018)

  4. Burr, G.W., Breitwisch, M.J., Franceschini, M., Garetto, D., Gopalakrishnan, K., Jackson, B., Kurdi, B., Lam, C., Lastras, L.A., Padilla, A., et al.: Phase change memory technology. J. Vac. Sci. Technol. B 28(2), 223–262 (2010)

    Article  Google Scholar 

  5. Chen, C., Yang, J., Wei, Q., Wang, C., Xue, M.: Fine-grained metadata journaling on NVM. In: Proceedings of IEEE Conference on MASS Storage Systems and Technologies (MSST), pp. 1–13 (2016)

  6. Chen, F., Mesnier, M.P., Hahn, S.: A protected block device for persistent memory. In: Proceedings of IEEE Conference on MASS Storage Systems and Technologies (MSST), pp. 1–12 (2014)

  7. Chidambaram, V., Pillai, T.S., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Optimistic crash consistency. In: Proceedings of the 24 ACM Symposium on Operating Systems Principles (SOSP), pp. 228–243 (2013)

  8. Chidambaram, V., Sharma, T., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Consistency without ordering. In: Proceedings of USENIX Conference on File and Storage Technologies (FAST), p. 9 (2012)

  9. Coburn, J., Caulfield, A.M., Akel, A., Grupp, L.M., Gupta, R.K., Jhala, R., Swanson, S.: Nv-heaps: making persistent objects fast and safe with next-generation, non-volatile memories. ACM Sigplan Not. 46(3), 105–118 (2011)

    Article  Google Scholar 

  10. Condit, J., Nightingale, E.B., Frost, C., Ipek, E., Lee, B., Burger, D., Coetzee, D.: Better i/o through byte-addressable, persistent memory. In: Proceedings of the 22nd symposium on Operating systems principles (SOSP), pp. 133–146 (2009)

  11. Corbet, J.: Supporting filesystems in persistent memory. https://lwn.net/Articles/610174/ (2014)

  12. Dulloor, S.R., Kumar, S., Keshavamurthy, A., Lantz, P., Reddy, D., Sankaran, R., Jackson, J.: System software for persistent memory. In: Proceedings of European Conference on Computer Systems (EuroSys), p. 15 (2014)

  13. ext4.org: Ext4. https://ext4.wiki.kernel.org (2016)

  14. Filebench: Filebench. https://github.com/filebench/filebench (2018)

  15. Intel: Intel. http://newsroom.intel.com/community (2015)

  16. Izraelevitz, J., Yang, J., Zhang, L., Kim, J., Liu, X., Memaripour, A., Soh, Y.J., Wang, Z., Xu, Y., Dulloor, S.R., et al.: Basic performance measurements of the intel optane dc persistent memory module. arXiv preprint arXiv:1903.05714 (2019)

  17. Kawahara, T.: Scalable spin-transfer torque ram technology for normally-off computing. IEEE Des. Test Comput. 1, 52–63 (2010)

    Google Scholar 

  18. Lantz, P., Rao, D.S., Kumar, S., Sankaran, R., Jackson, J.: Yat: A validation framework for persistent memory software. In: Proceedings of USENIX Technical Conference (ATC), pp. 433–438 (2014)

  19. Lee, B.C., Zhou, P., Yang, J., Zhang, Y., Zhao, B., Ipek, E., Mutlu, O., Burger, D.: Phase-change technology and the future of main memory. IEEE Micro 30(1), 143 (2010)

    Article  Google Scholar 

  20. Lee, E., Bahn, H., Noh, S.H.: Unioning of the buffer cache and journaling layers with non-volatile memory. In: Proceedings of USENIX Conference on File and Storage Technologies (FAST), pp. 73–80 (2013)

  21. Lee, E., Yoo, S., Jang, J.E., Bahn, H.: Shortcut-jfs: a write efficient journaling file system for phase change memory. In: Proceedings of IEEE Conference on MASS Storage Systems and Technologies (MSST), pp. 1–6. IEEE (2012)

  22. McKusick, M.K., Joy, W.N., Leffler, S.J., Fabry, R.S.: A fast file system for unix. ACM Trans. Comput. Syst. (TOCS) 2(3), 181–197 (1984)

    Article  Google Scholar 

  23. Moraru, I., Andersen, D.G., Kaminsky, M., Tolia, N., Ranganathan, P., Binkert, N.: Consistent, durable, and safe memory management for byte-addressable non volatile main memory. In: Proceedings of the First ACM SIGOPS Conference on Timely Results in Operating Systems, p. 1 (2013)

  24. Ou, J., Shu, J., Lu, Y.: A high performance file system for non-volatile main memory. In: Proceedings of European Conference on Computer Systems (EuroSys), p. 12 (2016)

  25. Oukid, I., Lasperas, J., Nica, A., Willhalm, T., Lehner, W.: Fptree: A hybrid SCM-DRAM persistent and concurrent b-tree for storage class memory. In: Proceedings of International Conference on Management of Data (SIGMOD), pp. 371–386 (2016)

  26. reiser4.org: Reiserfs. https://reiser4.wiki.kernel.org (2018)

  27. Rodeh, O., Bacik, J., Mason, C.: Btrfs: the linux b-tree filesystem. ACM Trans. Storage (TOS) 9(3), 9 (2013)

    Google Scholar 

  28. Rosenblum, M., Ousterhout, J.K.: The design and implementation of a log-structured file system. ACM Trans. Comput. Syst. (TOCS) 10(1), 26–52 (1992)

    Article  Google Scholar 

  29. Schwan, P., et al.: Lustre: Building a file system for 1000-node clusters. In: Proceedings of the 2003 Linux symposium, pp. 380–386 (2003)

  30. Sehgal, P., Basu, S., Srinivasan, K., Voruganti, K.: An empirical study of file systems on nvm. In: Proceedings of IEEE Conference on MASS Storage Systems and Technologies (MSST), pp. 1–14 (2015)

  31. Sweeney, A., Doucette, D., Hu, W., Anderson, C., Nishimoto, M., Peck, G.: Scalability in the XFS file system. In: Proceedings of USENIX Annual Technical Conference (ATC), vol. 15 (1996)

  32. Venkataraman, S., Tolia, N., Ranganathan, P., Campbell, R.H., et al.: Consistent and durable data structures for non-volatile byte-addressable memory. In: Proceedings of USENIX Conference on File and Storage Technologies (FAST), pp. 61–75 (2011)

  33. Volos, H., Tack, A.J., Swift, M.M.: Mnemosyne: Lightweight persistent memory. In: Proceedings of Sixteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 91–104 (2011)

  34. Wilcox, M.: Support ext4 on nv-dimm. https://lwn.net/Articles/588218/ (2014)

  35. Wu, M., Zwaenepoel, W.: envy: a non-volatile, main memory storage system. In: ACM SIGOPS Operating Systems Review, pp. 86–97 (1994)

  36. Wu, X., Reddy, A.: SCMFS: a file system for storage class memory. In: Proceedings of International Conference for High Performance Computing, Networking, Storage and Analysis (SC), p. 39 (2011)

  37. Xu, J., Swanson, S.: Nova: a log-structured file system for hybrid volatile/non-volatile main memories. In: Proceedings of USENIX Conference on File and Storage Technologies (FAST), pp. 323–338 (2016)

  38. Yang, J., Wei, Q., Chen, C., Wang, C., Yong, K.L., He, B.: NC-tree: Reducing consistency cost for NVM-based single level systems. In: Proceedings of USENIX Conference on File and Storage Technologies (FAST), pp. 167–181 (2015)

  39. Yang, J.J., Williams, R.S.: Memristive devices in computing system: promises and challenges. ACM J. Emerg. Technol. Computi. Syst. (JETC) 9(2), 11 (2013)

    Google Scholar 

  40. Zhang, X., Feng, D., Hua, Y., Chen, J.: Optimizing file systems with a write-efficient journaling scheme on non-volatile memory. IEEE Trans. Comput. 68(3), 402–413 (2018)

    Article  MathSciNet  Google Scholar 

  41. Zhang, Y., Yang, J., Memaripour, A., Swanson, S.: Mojim: a reliable and highly-available non-volatile memory system. In: Proceedings of Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pp. 3–18 (2015)

  42. Zhao, J., Li, S., Yoon, D.H., Xie, Y., Jouppi, N.P.: Kiln: Closing the performance gap between systems with and without persistence support. In: Proceedings of 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 421–432 (2013)

Download references

Funding

This work was supported by 2016YFB1000302 (National Key R&D Program of China), U1611261 (National Natural Science Foundation of China), 61433019 (National Natural Science Foundation of China), 61872392 (National Natural Science Foundation of China) and 2016ZT06D211 (Program for Guangdong Introducing Innovative and Entrepreneurial Teams).

Author information

Authors and Affiliations

  1. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China

    Yubo Liu, Hongbo Li, Yutong Lu, Zhiguang Chen & Nong Xiao

  2. School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, USA

    Yubo Liu & Ming Zhao

Authors
  1. Yubo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongbo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yutong Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiguang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ming Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yubo Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Li, H., Lu, Y. et al. HasFS: optimizing file system consistency mechanism on NVM-based hybrid storage architecture. Cluster Comput 23, 2501–2515 (2020). https://doi.org/10.1007/s10586-019-03023-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-019-03023-y

Keywords

Search

Navigation