Skip to main content
SpringerLink
Log in

Fine-grained checkpoint based on non-volatile memory

  • Published:
Frontiers of Information Technology & Electronic Engineering Aims and scope Submit manuscript

Abstract

New non-volatile memory (e.g., phase-change memory) provides fast access, large capacity, byte-addressability, and non-volatility features. These features, fast-byte-persistency, will bring new opportunities to fault tolerance. We propose a fine-grained checkpoint based on non-volatile memory. We extend the current virtual memory manager to manage non-volatile memory, and design a persistent heap with support for fast allocation and checkpointing of persistent objects. To achieve a fine-grained checkpoint, we scatter objects across virtual pages and rely on hardware page-protection to monitor the modifications. In our system, two objects in different virtual pages may reside on the same physical page. Modifying one object would not interfere with the other object. This allows us to monitor and checkpoint objects smaller than 4096 bytes in a fine-grained way. Compared with previous page-grained based checkpoint mechanisms, our new checkpoint method can greatly reduce the data copied at checkpoint time and better leverage the limited bandwidth of non-volatile memory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Badam, A., 2013. How persistent memory will change software systems. Computer, 46(8):45–51. http://dx.doi.org/10.1109/MC.2013.189

    Article  Google Scholar 

  • Bautista-Gomez, L., Tsuboi, S., Komatitsch, D., et al., 2011. FTI: high performance fault tolerance interface for hybrid systems. Proc. Int. Conf. for High Performance Computing, Networking, Storage and Analysis, Article 32. http://dx.doi.org/10.1145/2063384.2063427

    Google Scholar 

  • Bent, J., Gibson, G., Grider, G., et al., 2009. PLFS: a checkpoint filesystem for parallel applications. Proc. Int. Conf. for High Performance Computing, Networking, Storage and Analysis, Article 21. http://dx.doi.org/10.1145/1654059.1654081

    Google Scholar 

  • Berger, E.D., McKinley, K.S., Blumofe, R.D., et al., 2000. Hoard: a scalable memory allocator for multithreaded applications. ACM SIGPLAN Not., 35(11):117–128. http://dx.doi.org/10.1145/356989.357000

    Article  Google Scholar 

  • Cho, S., Lee, H., 2009. Flip-n-write: a simple deterministic technique to improve PRAM write performance, energy and endurance. Proc. 42nd Annual IEEE/ACM Int. Symp. on Microarchitecture, p.347–357.

    Google Scholar 

  • Chou, C., Jaleel, A., Qureshi, M.K., 2014. CAMEO: a two-level memory organization with capacity of main memory and flexibility of hardware-managed cache. Proc. 47th Annual IEEE/ACM Int. Symp. on Microarchitecture, p.1–12. http://dx.doi.org/10.1109/MICRO.2014.63

    Google Scholar 

  • Coburn, J., Caulfield, A.M., Akel, A., et al., 2011. NV-Heaps: making persistent objects fast and safe with nextgeneration, non-volatile memories. ACM SIGARCH Comput. Archit. News, 39(1):105–118. http://dx.doi.org/10.1145/1961295.1950380

    Article  Google Scholar 

  • Condit, J., Nightingale, E.B., Frost, C., et al., 2009. Better I/O through byte-addressable, persistent memory. Proc. ACM SIGOPS 22nd Symp. on Operating Systems Principles, p.133–146. http://dx.doi.org/10.1145/1629575.1629589

    Chapter  Google Scholar 

  • di Ventra, M., Pershin, Y.V., Chua, L.O., 2009. Circuit elements with memory: memristors, memcapacitors, and meminductors. Proc. IEEE, 97(10):1717–1724. http://dx.doi.org/10.1109/JPROC.2009.2021077

    Article  Google Scholar 

  • Dong, X., Xie, Y., Muralimanohar, N., et al., 2011. Hybrid checkpointing using emerging nonvolatile memories for future exascale systems. ACM Trans. Archit. Code Optim., 8(2), Article 6. http://dx.doi.org/10.1145/1970386.1970387

    Article  Google Scholar 

  • Dulloor, S.R., Kumar, S., Keshavamurthy, A., et al., 2014. System software for persistent memory. Proc. 9th European Conf. on Computer Systems, Article 15. http://dx.doi.org/10.1145/2592798.2592814

  • Felber, P., Fetzer, C., Riegel, T., 2008. Dynamic performance tuning of word-based software transactional memory. Proc. 13th ACM SIGPLAN Symp. on Principles and Practice of Parallel Programming, p.237–246. http://dx.doi.org/10.1145/1345206.1345241

    Google Scholar 

  • Felber, P., Fetzer, C., Marlier, P., et al., 2010. Time-based software transactional memory. IEEE Trans. Parall. Distr. Syst., 21(12):1793–1807. http://dx.doi.org/10.1109/TPDS.2010.49

    Article  Google Scholar 

  • Gulur, N., Mehendale, M., Manikantan, R., et al., 2014. Bimodal DRAM cache: improving hit rate, hit latency and bandwidth. Proc. 47th Annual IEEE/ACM Int. Symp. on Microarchitecture, p.38–50. http://dx.doi.org/10.1109/MICRO.2014.36

    Google Scholar 

  • Hirabayashi, M., 2010. Tokyo Cabinet: a Modern Implementation of DBM. http://fallabs.com/tokyocabinet/

    Google Scholar 

  • Jevdjic, D., Loh, G.H., Kaynak, C., et al., 2014. Unison cache: a scalable and effective die-stacked DRAM cache. Proc. 47th Annual IEEE/ACM Int. Symp. on Microarchitecture, p.25–37. http://dx.doi.org/10.1109/MICRO.2014.51

    Google Scholar 

  • Kannan, S., Gavrilovska, A., Schwan, K., et al., 2013. Optimizing checkpoints using NVM as virtual memory. Proc. IEEE 27th Int. Symp. on Parallel & Distributed Processing, p.29–40. http://dx.doi.org/10.1109/IPDPS.2013.69

    Google Scholar 

  • Koltsidas, I., Mueller, P., Pletka, R., et al., 2014. PSS: a prototype storage subsystem based on PCM. Proc. 5th Annual Non-volatile Memories Workshop, p.1–2.

    Google Scholar 

  • Lattner, C., Adve, V., 2004. LLVM: a compilation framework for lifelong program analysis & transformation. Proc. Int. Symp. on Code Generation and Optimization, p.75–86. http://dx.doi.org/10.1109/CGO.2004.1281665

    Google Scholar 

  • Li, D., Vetter, J.S., Marin, G., et al., 2012. Identifying opportunities for byte-addressable non-volatile memory in extreme-scale scientific applications. Proc. IEEE 26th Int. Parallel & Distributed Processing Symp., p.945–956. http://dx.doi.org/10.1109/IPDPS.2012.89

    Google Scholar 

  • Luk, C., Cohn, R., Muth, R., et al., 2005. Pin: building customized program analysis tools with dynamic instrumentation. ACM SIGPLAN Not., 40(6):190–200. http://dx.doi.org/10.1145/1064978.1065034

    Article  Google Scholar 

  • Minh, C., Chung, J., Kozyrakis, C., et al., 2008. STAMP: Stanford transactional applications for multi-processing. Proc. IEEE Int. Symp. on Workload Characterization, p.35–46. http://dx.doi.org/10.1109/IISWC.2008.4636089

    Google Scholar 

  • Plank, J.S., Li, K., Puening, M.A., 1998. Diskless checkpointing. IEEE Trans. Parall. Distr. Syst., 9(10):972–986. http://dx.doi.org/10.1109/71.730527

    Article  Google Scholar 

  • Qureshi, M.K., Srinivasan, V., Rivers, J.A., 2009. Scalable high performance main memory system using phasechange memory technology. ACM SIGARCH Comput. Archit. News, 37(3):24–33. http://dx.doi.org/10.1145/1555815.1555760

    Article  Google Scholar 

  • Qureshi, M.K., Franceschini, M.M., Jagmohan, A., et al., 2012. PreSET: improving performance of phase change memories by exploiting asymmetry in write times. ACM SIGARCH Comput. Archit. News, 40(3):380–391. http://dx.doi.org/10.1145/2366231.2337203

    Article  Google Scholar 

  • Rosenfeld, P., Cooper-Balis, E., Jacob, B., 2011. DRAMSim2: a cycle accurate memory system simulator. IEEE Comput. Archit. Lett., 10(1):16–19. http://dx.doi.org/10.1109/L-CA.2011.4

    Article  Google Scholar 

  • Schroeder, B., Gibson, G.A., 2007. Understanding failures in petascale computers. J. Phys. Conf. Ser., 78:012022. http://dx.doi.org/10.1088/1742-6596/78/1/012022

    Article  Google Scholar 

  • Volos, H., Tack, A.J., Swift, M.M., 2011. Mnemosyne: lightweight persistent memory. ACM SIGARCH Comput. Archit. News, 39(1):91–104. http://dx.doi.org/10.1145/1961295.1950379

    Article  Google Scholar 

  • Volos, H., Nalli, S., Panneerselvam, S., et al., 2014. Aerie: flexible file-system interfaces to storage-class memory. Proc. 9th European Conf. on Computer Systems, Article 14. http://dx.doi.org/10.1145/2592798.2592810

  • Wu, X., Reddy, A.L.N., 2011. SCMFS: a file system for storage class memory. Proc. Int. Conf. for High Performance Computing, Networking, Storage and Analysis, Article 39. http://dx.doi.org/10.1145/2063384.2063436

  • Xu, W., Sun, H., Wang, X., et al., 2011. Design of last-level on-chip cache using spin-torque transfer RAM (STT RAM). IEEE Trans. VLSI Syst., 19(3):483–493. http://dx.doi.org/10.1109/TVLSI.2009.2035509

    Article  Google Scholar 

  • Yoon, D.H., Muralimanohar, N., Chang, J., et al., 2011. FREE-p: protecting non-volatile memory against both hard and soft errors. Proc. IEEE 17th Int. Symp. on High Performance Computer Architecture, p.466–477. http://dx.doi.org/10.1109/HPCA.2011.5749752

    Google Scholar 

  • Zheng, G., Shi, L., Kale, L.V., 2004. FTC-Charm++: an in-memory checkpoint-based fault tolerant runtime for Charm++ and MPI. Proc. IEEE Int. Conf. on Cluster Computing, p.93–103. http://dx.doi.org/10.1109/CLUSTR.2004.1392606

    Google Scholar 

  • Zhou, P., Zhao, B., Yang, J., et al., 2009. A durable and energy efficient main memory using phase change memory technology. ACM SIGARCH Comput. Archit. News, 37(3):14–23. http://dx.doi.org/10.1145/1555815.1555759

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Science and Technology on Parallel and Distributed Processing Laboratory, College of Computer, National University of Defense Technology, Changsha, 410072, China

    Wen-zhe Zhang, Kai Lu, Xiao-ping Wang & Xu Zhou

  2. School of Computer, The University of Manchester, Manchester, M13 9PL, UK

    Mikel Luján

Authors
  1. Wen-zhe Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mikel Luján
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao-ping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Lu.

Additional information

Project supported by the National High-Tech R&D Program (863) of China (Nos. 2012AA01A301, 2012AA010901, 2012AA010303, and 2015AA01A301), the Program for New Century Excellent Talents in University, the National Natural Science Foundation of China (Nos. 61272142, 61402492, 61402486, 61379146, and 61272483), the Laboratory Pre-research Fund (No. 9140C810106150C81001), and the State Key Laboratory of High-End Server & Storage Technology (No. 2014HSSA01)

ORCID: Wen-zhe ZHANG, http://orcid.org/0000-0002-8798-2195

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Wz., Lu, K., Luján, M. et al. Fine-grained checkpoint based on non-volatile memory. Frontiers Inf Technol Electronic Eng 18, 220–234 (2017). https://doi.org/10.1631/FITEE.1500352

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1631/FITEE.1500352

Keywords

CLC number

Search

Navigation