Skip to main content

Advertisement

SpringerLink
Log in

Design of DRAM-NAND flash hybrid main memory and Q-learning-based prefetching method

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Owing to the increased need for machine learning and artificial intelligence in current cloud computing systems, the amount of data that needs to be processed has exponentially increased. Thus, it is important to optimize memory and storage systems to reduce the energy consumption and execution time of applications. This paper proposes a new Q-learning-based prefetching algorithm for DRAM–NAND flash hybrid main memory architecture. To minimize the computational overheads of learning-based schemes, we have designed two learning policies, namely aggressive learning and lazy learning. The proposed system reduces the energy consumption by about 80% of the memory and storage for Redis, OpenStack Swift which is a cloud computing open source framework and Apache Storm workloads. Further, the overall execution time of workloads in cloud computing applications is reduced by almost half. Using a path generator with a Q-learning-based prefetching algorithm, we realize an increased hit rate of about 21% compared to that with a no-prefetching system, compared to non-prefetching system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Fitzpatrick B (2004) Distributed caching with memcached. Linux J 2004(124):5

    Google Scholar 

  2. Zawodny J (2009) Redis: lightweight key/value store that goes the extra mile. Linux Mag 79

  3. Qureshi MK, Srinivasan V, Riversm JA (2009) Scalable high performance main memory system using phase-change memory technology. In: The 36th Annual International Symposium on Computer Architecture, pp 24–33

  4. Loh GH (2008) 3D-stacked memory architectures for multi-core processors. In: The 35th Annual International Symposium on Computer Architecture, pp 453–464

  5. Nishtala R, Fugal H, Grimm S, Kwiatkowski M, M, Lee H, Li HC, et al (2013) Scaling memcache at Facebook. In: The 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13), pp 385–398

  6. Lee DH, Yoon SK, Kim JG, Weems CC, Kim SD (2015) A new memory-disk integrated system with HW optimizer. ACM Trans Archit Code Optim 12(2):11

    Article  Google Scholar 

  7. Dhiman G, Ayoub GR, Rosing T (2009) PDRAM: a hybrid PRAM and DRAM main memory system. In: The 46th ACM/IEEE Design Automation Conference (DAC’09), pp 664–669

  8. Kgil T, Roberts D, Mudge T (2008) Improving NAND flash based disk caches. In: The 35th Annual International Symposium on Computer Architecture, pp 327–338

  9. Anand A, Muthukrishnan C, Kappes S, Akella A, Nath S (2010) Cheap and large CAMs for high performance data-intensive networked systems. NSDI 10:29

    Google Scholar 

  10. Cidon A, Eisenman A, Alizadeh M, Katti S (2015) Dynacache: dynamic cloud caching. In: The 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 15)

  11. Byan S, Lentini J, Madan A, Pabon L (2012) Mercury: host-side flash caching for the data center. In: IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST), pp 1–12

  12. Chen Z, Lu Y, Xiao N, Liu F (2014) A hybrid memory built by SSD and DRAM to support in-memory big data analytics. Knowl Inf Syst 41(2):335–354

    Article  Google Scholar 

  13. Kgil T, Mudge T (2006) FlashCache: a NAND flash memory file cache for low power web servers. In: The 2006 International Conference on Compilers, Architecture and Synthesis for Embedded Systems, pp 103–112

  14. Arteaga D, Cabrera J, Xu J, Sundararaman S, Zhao M (2016) CloudCache: on-demand flash cache management for cloud computing. In: The 14th USENIX Conference on File and Storage Technologies (FAST 16), pp 355–369

  15. Smith AJ (1978) Sequential program prefetching in memory hierarchies. Computer 11(12):721

    Article  Google Scholar 

  16. Li Z, Chen Z, Srinivasan SM, Zhou Y (2004) C-miner: mining block correlations in storage systems. FAST 4:173–186

    Google Scholar 

  17. Arnold J (2014) OpenStack swift: using, administering, and developing for swift object storage. O’Reilly Media, Newton

    Google Scholar 

  18. Kim J et al (2016) Path confidence based lookahead prefetching. In: 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE

  19. Michaud P (2016) Best-offset hardware prefetching. In: 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, pp 469–480

  20. Jain A, Lin C (2013) Linearizing irregular memory accesses for improved correlated prefetching. In: 2013 46th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, pp 247–259

  21. Ishii Y, Inaba M, Hiraki K (2011) Access map pattern matching for high performance data cache prefetch. J Instr Level Parallelism 13:1–24

    Google Scholar 

  22. Mohan V (2014) A comparison of various reinforcement learning algorithms to solve racetrack problem

  23. Arabnejad H et al (2017) A comparison of reinforcement learning techniques for fuzzy cloud auto-scaling. In: 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID). IEEE

  24. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292

    MATH  Google Scholar 

  25. Hu Z, Martonosi M, Kaxiras S (2003) TCP: tag correlating prefetchers. In: The 9th International Symposium on High-performance Computer Architecture, pp 317–326

  26. Joseph D, Grunwald D (1999) Prefetching using markov predictors. IEEE Trans Comput 48(2):121–133

    Article  Google Scholar 

  27. Kandiraju GB, Sivasubramaniam A (2002) Going the distance for TLB prefetching: an application-driven study. IEEE Comput Soc 30(2):195–206

    Google Scholar 

  28. Song W, Kim Y, Kim H, Lim J, Kim J (2014) Personalized optimization for android smartphones. ACM Trans Embed Comput Syst (TECS) 13(2s):60

    Google Scholar 

  29. Nesbit KJ, Smith JE (2004) Data cache prefetching using a global history buffer. In: International Symposium on High Performance Computer Architecture, pp 96–105

  30. Kim S, Veidenbaum AV (1997) Stride-directed prefetching for secondary caches. In: The 1997 International Conference on Parallel Processing, pp 314–321

  31. Kryder M, Kim C (2010) After hard drives what comes next? The IEEE Trans Magn 45:3406–3413

    Article  Google Scholar 

  32. Park C, Kang JU, Park SY, Kim JS (2004) Energy-aware Demand Paging on NAND Flash-based Embedded Storages. In: The 2004 International Symposium on Low Power Electronics and Design, pp 338–343

  33. Chang LP (2008) Hybrid Solid-state Disks: combining Heterogeneous NAND Flash in Large SSDs. In: The 2008 Asia and South Pacific Design Automation Conference, pp 428–433

  34. Bellard F (2005) QEMU: a fast and portable dynamic translator. In: USENIX Annual Technical Conference (FREENIX Track), pp 41–46

  35. Kim Y, Tauras B, Gupta A, Urgaonkar B (2009) Flashsim: a simulator for NAND flash-based solid-state drives. In: The First International Conference on Advances in System Simulation, pp 125–131

  36. Cooper BF, Silberstein A, Tam E, Ramakrishnan R, Sears R (2010) Benchmarking cloud serving systems with YCSB. In: The 1st ACM Symposium on Cloud Computing, pp 143–154

  37. Hunt P, Konar M, Junqueira FP, Reed B (2010) ZooKeeper: wait-free coordination for internet-scale systems. In: USENIX Annual Technical Conference. vol 8, p 9

  38. Shao Z, Liu Y, Chen Y, Li T (2012) Utilizing PCM for Energy optimization in embedded systems. In: 2012 IEEE Computer Society Annual Symposium on VLSI, pp 398–403

  39. Shuja J et al (2016) Survey of techniques and architectures for designing energy-efficient data centers. IEEE Syst J 10(2):507–519

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by Next-Generation Information Computing Development Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT Future Planning (NRF-2015M3C4A7065522).

Author information

Authors and Affiliations

  1. Yonsei University, Seoul, South Korea

    Su-Kyung Yoon, Young-Sun Youn, Jeong-Geun Kim & Shin-Dug Kim

Authors
  1. Su-Kyung Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Young-Sun Youn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeong-Geun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shin-Dug Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shin-Dug Kim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yoon, SK., Youn, YS., Kim, JG. et al. Design of DRAM-NAND flash hybrid main memory and Q-learning-based prefetching method. J Supercomput 74, 5293–5313 (2018). https://doi.org/10.1007/s11227-018-2421-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2421-7

Keywords

Search

Navigation