Skip to main content
Springer Nature Link
Log in

Lightweight and Accurate Memory Allocation in Key-Value Cache

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

The use of key-value caches in modern web servers is becoming more and more ubiquitous. Representatively, Memcached as a widely used key-value cache system, originally intended for speeding up dynamic web applications by alleviating database load. One of the key factors affecting the performance of Memcached is the memory allocation among different item classes. How to obtain the most efficient partitioning scheme with low time and space consumption is a focus of attention. In this paper, we propose a lightweight and accurate memory allocation scheme in Memcached, by sampling access patterns, analyzing data locality, and reassigning the memory space. One early study on optimizing memory allocation is LAMA, which uses footprint-based MRC to optimize memory allocation in Memcached. However, LAMA does not model deletion operations in Memcached and its spatial overhead is quite large. We propose a method that consumes only 3% of LAMA space and can handle read, write and deletion operations. Moreover, evaluation results show that the average stable-state miss ratio is reduced by 15.0% and the average stable-state response time is reduced by 12.3% when comparing our method to LAMA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Redis website. https://redis.io/ (2018). Accessed: 2018-07-10

  2. Memcached website. http://memcached.org/ (2018). Accessed: 2018-07-10

  3. Atikoglu, B., Xu, Y., Frachtenberg, E., Jiang, S., Paleczny, M.: Workload analysis of a large-scale key-value store. In: ACM SIGMETRICS Performance Evaluation Review, vol. 40, pp. 53–64. ACM (2012)

  4. Cidon, A., Rushton, D., Rumble, S.M., Stutsman, R.: Memshare: a dynamic multi-tenant key-value cache. In: 2017 USENIX Annual Technical Conference (USENIX ATC 17), pp. 321–334. USENIX Association, Santa Clara (2017). https://www.usenix.org/conference/atc17/technical-sessions/presentation/cidon

  5. Caching with twemcache. https://blog.twitter.com/engineering/en_us/a/2012/caching-with-twemcache.html (2018). Accessed: 2018-07-10

  6. Hart, S., Frachtenberg, E., Berezecki, M.: Predicting Memcached throughput using simulation and modeling. TMS/DEVS’12, pp. 40:1–40:8. http://dl.acm.org/citation.cfm?id=2346616.2346656

  7. Saemundsson, T., Bjornsson, H., Chockler, G., Vigfusson, Y.: Dynamic performance profiling of cloud caches. In: SOCC’14, pp. 28:1–28:14. https://doi.org/10.1145/2670979.2671007

  8. Jose, J., Subramoni, H., Kandalla, K., Wasi-ur Rahman, M., Wang, H., Narravula, S., Panda, D.K.: Scalable memcached design for InfiniBand clusters using hybrid transports. In: CCGRID’12, pp. 236–243. https://doi.org/10.1109/CCGrid.2012.141

  9. Fan, B., Andersen, D.G., Kaminsky, M.: Memc3: Compact and concurrent Memcache with dumber caching and smarter hashing. In: NSDI’13, pp. 371–384. https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/fan

  10. Lim, K., Meisner, D., Saidi, A.G., Ranganathan, P., Wenisch, T.F.: Thin servers with smart pipes: designing soc accelerators for Memcached. ISCA’13. https://doi.org/10.1145/2485922.2485926

  11. Hwang, J., Wood, T.: Adaptive performance-aware distributed memory caching. In: ICAC’13, pp. 33–43. https://www.usenix.org/conference/icac13/technical-sessions/presentation/hwang

  12. Zhang, W., Hwang, J., Wood, T., Ramakrishnan, K., Huang, H.: Load balancing of heterogeneous workloads in memcached clusters. In: Feedback Computing’14. https://www.usenix.org/conference/feedbackcomputing14/workshop-program/presentation/zhang

  13. Twemcache. https://twitter.com/twemcache (2018). Accessed: 2018-07-10

  14. Nishtala, R., Fugal, H., Grimm, S., Kwiatkowski, M., Lee, H., Li, H.C., McElroy, R., Paleczny, M., Peek, D., Saab, P., Stafford, D., Tung, T., Venkataramani, V.: Scaling Memcache at facebook. In: NSDI’13, pp. 385–398. https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/nishtala

  15. Hu, X., Wang, X., Li, Y., Zhou, L., Luo, Y., Ding, C., Jiang, S., Wang, Z.: LAMA: Optimized locality-aware memory allocation for key-value cache. In: USENIX ATC’15, pp. 57–69. https://www.usenix.org/conference/atc15/technical-session/presentation/hu

  16. Hu, X., Wang, X., Zhou, L., Luo, Y., Ding, C., Jiang, S., Wang, Z.: Optimizing locality-aware memory management of key-value caches. IEEE Trans. Comput. 66(5), 862–875 (2017). https://doi.org/10.1109/TC.2016.2618920

    Article  MathSciNet  Google Scholar 

  17. Xiang, X., Bao, B., Ding, C., Gao, Y.: Linear-time modeling of program working set in shared cache. In: PACT’11, pp. 350–360

  18. Hu, X., Wang, X., Zhou, L., Luo, Y., Ding, C., Wang, Z.: Kinetic modeling of data eviction in cache. In: USENIX ATC’16

  19. Byrne, D., Onder, N., Wang, Z.: mPart: miss-ratio curve guided partitioning in key-value stores. In: Proceedings of the 2018 ACM SIGPLAN International Symposium on Memory Management, ISMM 2018, pp. 84–95. ACM, New York (2018). https://doi.org/10.1145/3210563.3210571

  20. Hu, X., Wang, X., Zhou, L., Luo, Y., Wang, Z., Ding, C., Ye, C.: Fast miss ratio curve modeling for storage cache. ACM Trans. Storage 14(2), 12:1–12:34 (2018). https://doi.org/10.1145/3185751

    Article  Google Scholar 

  21. Pan, C., Hu, X., Zhou, L., Luo, Y., Wang, X., Wang, Z.: PACE: penalty aware cache modeling with enhanced AET. In: APSys’18. ACM (2018). https://doi.org/10.1145/3265723.3265736

  22. Mutilate. https://github.com/leverich/mutilate (2018). Accessed: 2018-07-10

  23. Mattson, R.L., Gecsei, J., Slutz, D.R., Traiger, I.L.: Evaluation techniques for storage hierarchies. IBM Syst. J. 9(2), 78–117 (1970). https://doi.org/10.1147/sj.92.0078

    Article  MATH  Google Scholar 

  24. Niu, Q., Dinan, J., Lu, Q., Sadayappan, P.: Parda: a fast parallel reuse distance analysis algorithm. IPDPS’12, pp. 1284–1294. https://doi.org/10.1109/IPDPS.2012.117

  25. Wires, J., Ingram, S., Drudi, Z., Harvey, N.J.A., Warfield, A.: Characterizing storage workloads with counter stacks. In: OSDI’14, pp. 335–349. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/wires

  26. Waldspurger, C.A., Park, N., Garthwaite, A., Ahmad, I.: Efficient MRC construction with SHARDS. In: FAST’15, pp. 95–110

  27. Suh, G.E., Devadas, S., Rudolph, L.: Analytical cache models with applications to cache partitioning. In: ICS’01, pp. 1–12

  28. Zhang, X., Dwarkadas, S., Shen, K.: Towards practical page coloring-based multicore cache management. In: EuroSys’09, pp. 89–102

  29. Cascaval, C., Duesterwald, E., Sweeney, P.F., Wisniewski, R.W.: Multiple page size modeling and optimization. In: PSCT’05, pp. 339–349

  30. Zhou, P., Pandey, V., Sundaresan, J., Raghuraman, A., Zhou, Y., Kumar, S.: Dynamic tracking of page miss ratio curve for memory management. ACM SIGOPS Oper. Syst. Rev. 38, 177–188 (2004)

    Article  Google Scholar 

  31. Kim, Y.H., Hill, M.D., Wood, D.A.: Implementing stack simulation for highly-associative memories. In: SIGMETRICS’91, pp. 212–213

  32. Bjornsson, H., Chockler, G., Saemundsson, T., Vigfusson, Y.: Dynamic performance profiling of cloud caches. In: SOCC’13

Download references

Acknowledgements

We would like to thank anonymous reviewers for their constructive comments and suggestions. The research is supported in part by the National Science Foundation of China (Nos. 61472008, 61672053 and U1611461), Shenzhen Key Research Project No. JCYJ20170412150946024, National Science Foundation CSR1618384 and the National Key R&D Program of China under Grant No. 2018YFB1003505.

Author information

Authors and Affiliations

  1. Department of Computer Science, Institute of Network Computing and Information Systems, Peking University, Beijing, China

    Cheng Pan, Lan Zhou, Yingwei Luo & Xiaolin Wang

  2. Shenzhen Key Lab for Information Centric Networking and Blockchain Technology (ICNLAB), SECE, Peking University, Shenzhen, China

    Cheng Pan & Yingwei Luo

  3. Department of Computer Science, Michigan Technological University, Houghton, MI, USA

    Zhenlin Wang

Authors
  1. Cheng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingwei Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaolin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhenlin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cheng Pan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pan, C., Zhou, L., Luo, Y. et al. Lightweight and Accurate Memory Allocation in Key-Value Cache. Int J Parallel Prog 47, 451–466 (2019). https://doi.org/10.1007/s10766-018-0616-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-018-0616-4

Keywords