Skip to main content
SpringerLink
Log in

DAOS Beyond Persistent Memory: Architecture and Initial Performance Results

  • Conference paper
  • First Online:
High Performance Computing (ISC High Performance 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13999))

Included in the following conference series:

Abstract

The Distributed Asynchronous Object Storage (DAOS) is an open source scale-out storage system that is designed from the ground up to support Storage Class Memory (SCM) and NVMe storage in user space. Until now, the DAOS storage stack has been based on Intel Optane Persistent Memory (PMem) and the Persistent Memory Development Kit (PMDK). With the discontinuation of Optane PMem, and no persistent CXL.mem devices in the market yet, DAOS continues to support PMem-based servers but now also supports server configurations where its Versioning Object Store (VOS) is held in DRAM. In this case, the VOS data structures are persisted through a synchronous Write-Ahead-Log (WAL) combined with asynchronous checkpointing to NVMe SSDs. This paper describes the new non-PMem DAOS architecture, and reports first performance results based on a DAOS 2.4 technology preview.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Liang, Z., Lombardi, J., Chaarawi, M., Hennecke, M.: DAOS: a scale-out high performance storage stack for storage class memory. In: Panda, D.K. (ed.) SCFA 2020. LNCS, vol. 12082, pp. 40–54. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-48842-0_3

    Chapter  Google Scholar 

  2. Liang, Z., Fan, Y., Wang, D., Lombardi, J.: Distributed transaction and self-healing system of DAOS. In: Nichols, J., Verastegui, B., Maccabe, A.‘, Hernandez, O., Parete-Koon, S., Ahearn, T. (eds.) SMC 2020. CCIS, vol. 1315, pp. 334–348. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63393-6_22

    Chapter  Google Scholar 

  3. Scot Breitenfeld, M., et al.: DAOS for extreme-scale systems in scientific applications (2017). https://arxiv.org/pdf/1712.00423.pdf

  4. IO500. https://io500.org/

  5. Rudoff, A.: APIs for persistent memory programming (2018). https://storageconference.us/2018/Presentations/Rudoff.pdf

  6. Scargall, S.: Programming Persistent Memory. Apress, Berkeley ISBN 978-1-4842-4931-4 (2020). https://doi.org/10.1007/978-1-4842-4932-1

  7. PMDK. https://pmem.io/pmdk/

  8. SPDK. https://spdk.io/

  9. Intel® Optane™ Persistent Memory 200 Series. https://ark.intel.com/content/www/us/en/ark/products/series/203877/intel-optane-persistent-memory-200-series.html

  10. Support for Intel® Optane™ Persistent Memory 200 Series. https://www.intel.com/content/www/us/en/support/products/203877/memory-and-storage/intel-optane-persistent-memory/intel-optane-persistent-memory-200-series.html

  11. Customer Support Options for Discontinued Intel Optane Solid-State Drives and Modules. https://www.intel.com/content/www/us/en/support/articles/000024320.html

  12. DAOS vos_on_blob Feature Branch (build daos-2.3.106-2.9536.ge3c942ec.el8.x86_64). https://github.com/daos-stack/daos/tree/feature/vos_on_blob

  13. Lombardi, J., et al.: Metadata on SSDs design documentation (2022). https://daosio.atlassian.net/wiki/spaces/DC/pages/11196923911/Metadata+on+SSDs

  14. Niu, Y.: WAL detailed design (2022). https://daosio.atlassian.net/wiki/spaces/DC/pages/11215339529/WAL+Detailed+Design

  15. Hennecke, M.: Performance evolution of DAOS servers (2023). https://www.intel.com/content/www/us/en/high-performance-computing/performance-evolution-of-daos-servers.html

  16. IOR and mdtest github repository. https://github.com/hpc/ior

  17. The DFS (DAOS File System) driver. https://github.com/hpc/ior/blob/main/README_DAOS

  18. Solidigm/Intel SSD D7-P5510 Product Brief. https://www.solidigm.com/content/dam/solidigm/en/site/products/data-center/d7/p5510/documents/d7-p5510-series-product-brief.pdf

  19. Hennecke, M.: Understanding DAOS storage performance scalability. In: International Conference on High Performance Computing in Asia-Pacific Region Workshops (HPCASIAWORKSHOP 2023), 27 February–2 March 2023, Raffles Blvd, Singapore (2023). https://doi.org/10.1145/3581576.3581577

  20. Migration from Direct-Attached Intel Optane Persistent Memory to CXL-Attached Memory. https://www.intel.com/content/www/us/en/products/docs/memory-storage/optane-persistent-memory-to-cxl-attached-memory.html

Download references

Author information

Authors and Affiliations

  1. Intel Deutschland GmbH, Am Campeon 10, 85579, Neubiberg, Germany

    Michael Hennecke

  2. Intel Corporation, 2200 Mission College Blvd., Santa Clara, CA, 95054-1549, USA

    Jeff Olivier

  3. Intel Corporation (UK) Ltd., Pipers Way, Swindon, SN3 1RJ, UK

    Tom Nabarro

  4. Intel China Ltd., GTC, No. 36 3Rd Ring Road, Beijing, China

    Liang Zhen, Yawei Niu, Shilong Wang & Xuezhao Liu

Authors
  1. Michael Hennecke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeff Olivier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tom Nabarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liang Zhen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yawei Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shilong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xuezhao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Hennecke .

Editor information

Editors and Affiliations

  1. University of New Mexico, Albuquerque, NM, USA

    Amanda Bienz

  2. University of Edinburgh, Edinburgh, UK

    Michèle Weiland

  3. Université Paris-Saclay, Gif sur Yvette, France

    Marc Baboulin

  4. CERFACS, Toulouse, France

    Carola Kruse

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hennecke, M. et al. (2023). DAOS Beyond Persistent Memory: Architecture and Initial Performance Results. In: Bienz, A., Weiland, M., Baboulin, M., Kruse, C. (eds) High Performance Computing. ISC High Performance 2023. Lecture Notes in Computer Science, vol 13999. Springer, Cham. https://doi.org/10.1007/978-3-031-40843-4_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-40843-4_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-40842-7

  • Online ISBN: 978-3-031-40843-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation