Skip to main content
SpringerLink
Log in

From Static to Malleable: Improving Flexibility and Compatibility in Burst Buffer File Systems

  • Conference paper
  • First Online:
High Performance Computing (ISC High Performance 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13999))

Included in the following conference series:

  • 1027 Accesses

Abstract

Numerous burst buffer file systems have been developed in recent years in the context of high-performance computing (HPC). These file systems aim to combat I/O bottlenecks and cross-application interference that arise in parallel file systems due to competing and uncoordinated I/O patterns. While burst buffer file systems have demonstrated linear scaling for metadata and data workloads, their configurations are currently static and cannot adapt to changing requirements from applications or the HPC system. User space file system implementations have emerged as a promising solution since they can be customized to meet specific application I/O requirements. However, developing file systems in user space comes with significant challenges, such as the lack of mature I/O interfaces that can hurt application compatibility.

This paper explores the challenges that have shaped the design of GekkoFS, an exemplary user space burst buffer file system, and presents an overhauled file system architecture to improve application compatibility and provide a foundation for additional malleability techniques, molding the file system to the application’s requirements. We also evaluate this new architecture and show its performance overhead that remains well above the storage capability of commonly used node-local storage devices. Overall, the paper demonstrates the potential benefits of this architecture for user space file system implementations and presents a path forward for their continued development.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.admire-eurohpc.eu/.

References

  1. Fuse (filesystem in userspace) (2002). https://github.com/libfuse/libfuse

  2. Persistent memory development kit (2014). https://pmem.io/pmdk/

  3. Ior and mdtest (2020). https://github.com/hpc/ior

  4. BeeGFS: Beeond: Beegfs on demand (2018). https://www.beegfs.io/wiki/BeeOND

  5. Brinkmann, A., et al.: Ad hoc file systems for high-performance computing. J. Comput. Sci. Technol. 35(1), 4–26 (2020). https://doi.org/10.1007/s11390-020-9801-1

    Article  Google Scholar 

  6. Didona, D., Pfefferle, J., Ioannou, N., Metzler, B., Trivedi, A.: Understanding modern storage APIs: a systematic study of libaio, SPDK, and io_uring. In: SYSTOR 2022: The 15th ACM International Systems and Storage Conference. ACM (2022)

    Google Scholar 

  7. Dilger, A.: Lustre metadata scaling (2012)

    Google Scholar 

  8. Dun, N., Taura, K., Yonezawa, A.: GMount: an ad hoc and locality-aware distributed file system by using SSH and FUSE. In: 9th IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid) (2009)

    Google Scholar 

  9. Fischer, P., Lottes, J., Tufo, H.: Nek5000. Technical report, Argonne National Lab. (ANL), Argonne, IL, United States (2007)

    Google Scholar 

  10. Folk, M., Heber, G., Koziol, Q., Pourmal, E., Robinson, D.: An overview of the HDF5 technology suite and its applications. In: Proceedings of the 2011 EDBT/ICDT Workshop on Array Databases. ACM (2011)

    Google Scholar 

  11. Garcia-Blas, J., Singh, D.E., Carretero, J.: IMSS: in-memory storage system for data intensive applications. In: Anzt, H., Bienz, A., Luszczek, P., Baboulin, M. (eds.) ISC High Performance 2022. LNCS, vol. 13387, pp. 190–205. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-23220-6_13

    Chapter  Google Scholar 

  12. Haynes, T.: Network file system (NFS) version 4 minor version 2 protocol. RFC 7862 (2016)

    Google Scholar 

  13. Huai, Q., Hsu, W., Lu, J., Liang, H., Xu, H., Chen, W.: XFUSE: an infrastructure for running filesystem services in user space. In: 2021 USENIX Annual Technical Conference, USENIX ATC 2021. USENIX Association (2021)

    Google Scholar 

  14. Isaila, F., Blas, J.G., Carretero, J., Liao, W., Choudhary, A.N.: AHPIOS: an MPI-based ad hoc parallel I/O system. In: 14th International Conference on Parallel and Distributed Systems (ICPADS) (2008)

    Google Scholar 

  15. L.L.N. Laboratory: Unifyfs (2019). https://github.com/LLNL/UnifyFS

  16. Lensing, P.H., Cortes, T., Brinkmann, A.: Direct lookup and hash-based metadata placement for local file systems. In: 6th Annual International Systems and Storage Conference, SYSTOR 2013. ACM (2013)

    Google Scholar 

  17. Miranda, A., Effert, S., Kang, Y., Miller, E.L., Brinkmann, A., Cortes, T.: Reliable and randomized data distribution strategies for large scale storage systems. In: 18th International Conference on High Performance Computing. IEEE (2011)

    Google Scholar 

  18. Moti, N., et al.: Simurgh: a fully decentralized and secure NVMM user space file system. In: SC 2021: The International Conference for High Performance Computing, Networking, Storage and Analysis. ACM (2021)

    Google Scholar 

  19. Poliakoff, D., LeGendre, M.: Gotcha: an function-wrapping interface for HPC tools. In: Bhatele, A., Boehme, D., Levine, J.A., Malony, A.D., Schulz, M. (eds.) ESPT/VPA 2017-2018. LNCS, vol. 11027, pp. 185–197. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17872-7_11

    Chapter  Google Scholar 

  20. Soumagne, J., et al.: Mercury: enabling remote procedure call for high-performance computing. In: IEEE International Conference on Cluster Computing, CLUSTER (2013)

    Google Scholar 

  21. Tatebe, O., Obata, K., Hiraga, K., Ohtsuji, H.: CHFS: parallel consistent hashing file system for node-local persistent memory. In: HPC Asia 2022: International Conference on High Performance Computing in Asia-Pacific Region (2022)

    Google Scholar 

  22. Vangoor, B.K.R., Tarasov, V., Zadok, E.: To FUSE or not to FUSE: performance of user-space file systems. In: 15th USENIX Conference on File and Storage Technologies (FAST) (2017)

    Google Scholar 

  23. Vef, M., et al.: GekkoFS - a temporary burst buffer file system for HPC applications. J. Comput. Sci. Technol. 35(1), 72–91 (2020). https://doi.org/10.1007/s11390-020-9797-6

    Article  Google Scholar 

  24. Vef, M., et al.: GekkoFS - a temporary distributed file system for HPC applications. In: IEEE International Conference on Cluster Computing, CLUSTER 2018, Belfast, UK, 10–13 September 2018. IEEE Computer Society (2018)

    Google Scholar 

  25. Vienne, J.: Benefits of cross memory attach for MPI libraries on HPC clusters. In: Annual Conference of the Extreme Science and Engineering Discovery Environment, XSEDE 2014, Atlanta, GA, USA, 13–18 July 2014, pp. 33:1–33:6 (2014)

    Google Scholar 

  26. Wang, C.: Detecting data races on relaxed systems using recorder (2022)

    Google Scholar 

  27. Wang, C., Mohror, K., Snir, M.: File system semantics requirements of HPC applications. In: HPDC 2021: The 30th International Symposium on High-Performance Parallel and Distributed Computing, Virtual Event. ACM (2021)

    Google Scholar 

  28. Wang, T., Mohror, K., Moody, A., Sato, K., Yu, W.: An ephemeral burst-buffer file system for scientific applications. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2016 (2016)

    Google Scholar 

  29. Zhu, Y., et al.: Direct-FUSE: removing the middleman for high-performance FUSE file system support. In: Proceedings of the 8th International Workshop on Runtime and Operating Systems for Supercomputers, ROSS@HPDC 2018. ACM (2018)

    Google Scholar 

Download references

Acknowledgements

This research was conducted using the supercomputer Mogon II and services offered by Johannes Gutenberg University Mainz. The authors gratefully acknowledge the computing time granted on Mogon II.

This work was partially funded by the European Union’s Horizon 2020 and the German Ministry of Education and Research (BMBF) under the “Adaptive multi-tier intelligent data manager for Exascale (ADMIRE)” project; Grant Agreement number: 956748-ADMIRE-H2020-JTI-EuroHPC-2019-1. Further, this work was partially supported by the Spanish Ministry of Economy and Competitiveness (MINECO) under grants PID2019-107255GB, and the Generalitat de Catalunya under contract 2021-SGR-00412. This publication is part of the project ADMIRE PCI2021-121952, funded by MCIN/AEI/10.13039/501100011033.

Author information

Authors and Affiliations

  1. Johannes Gutenberg University Mainz, Mainz, Germany

    Marc-André Vef & André Brinkmann

  2. Barcelona Supercomputing Center, Barcelona, Spain

    Alberto Miranda & Ramon Nou

Authors
  1. Marc-André Vef
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto Miranda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramon Nou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. André Brinkmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marc-André Vef .

Editor information

Editors and Affiliations

  1. University of New Mexico, Albuquerque, NM, USA

    Amanda Bienz

  2. University of Edinburgh, Edinburgh, UK

    Michèle Weiland

  3. Université Paris-Saclay, Gif sur Yvette, France

    Marc Baboulin

  4. CERFACS, Toulouse, France

    Carola Kruse

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vef, MA., Miranda, A., Nou, R., Brinkmann, A. (2023). From Static to Malleable: Improving Flexibility and Compatibility in Burst Buffer File Systems. In: Bienz, A., Weiland, M., Baboulin, M., Kruse, C. (eds) High Performance Computing. ISC High Performance 2023. Lecture Notes in Computer Science, vol 13999. Springer, Cham. https://doi.org/10.1007/978-3-031-40843-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-40843-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-40842-7

  • Online ISBN: 978-3-031-40843-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation