ABSTRACT
Due to large data volume and low latency requirements of modern web services, the use of in-memory key-value (KV) cache often becomes an inevitable choice (e.g. Redis and Memcached). The in-memory cache holds hot data, reduces request latency, and alleviates the load on background databases. Inheriting from the traditional hardware cache design, many existing KV cache systems still use recency-based cache replacement algorithms, e.g., LRU or its approximations. However, the diversity of miss penalty distinguishes a KV cache from a hardware cache. Inadequate consideration of penalty can substantially compromise space utilization and request service time. KV accesses also demonstrate locality, which needs to be coordinated with miss penalty to guide cache management.
We propose pRedis, Penalty and Locality Aware Memory Allocation in Redis, which synthesizes data locality and miss penalty, in a quantitative manner, to guide memory allocation and replacement in Redis. At the same time, we also explore the diurnal behavior of a KV store and exploit long-term reuse. We replace the original passive eviction mechanism with an automatic dump/load mechanism, in order to smooth the transition between access peaks and valleys. Our evaluation shows that pRedis effectively reduces the average and tail access latency with minimal time and space overhead. For both real-world and synthetic workloads, our approach delivers an average of 14% ~ 52% latency reduction over a state-of-the-art penalty aware cache management scheme, Hyperbolic Caching, and shows more quantitative predictability of performance.
- Approximated LRU. https://redis.io/topics/lru-cache. Accessed: 2010-06-10.Google Scholar
- Memcached Memory Management Blog. https://www.loginradius.com/engineering/memcach-memory-management. Accessed: 2019-06-10.Google Scholar
- Memcached Website. http://memcached.org. Accessed: 2019-06-10.Google Scholar
- Memtier_benchmark. https://github.com/GarantiaData\/memtier_benchmark. Accessed: 2019-06-10.Google Scholar
- MSR Cambridge Traces. http://iotta.snia.org/traces/388. Accessed: 2019-06-10.Google Scholar
- MurmurHash. https://en.wikipedia.org/wiki/Murmur-\Hash. Accessed: 2019-06-10.Google Scholar
- MySQL Website. https://www.mysql.com. Accessed: 2019-06-10.Google Scholar
- MySQLslap. https://tosbourn.com/mysqlslap-a-quickstart-guide/. Accessed: 2019-06-10.Google Scholar
- PostgreSQL Website. https://www.postgresql.org/. Accessed: 2019-06-10.Google Scholar
- Redis as an LRU cache. http://oldblog.antirez.com/post/redis-as-LRU-cache.html. Accessed: 2019-06-10.Google Scholar
- Redis Website. https://redis.io. Accessed: 2019-06-10.Google Scholar
- Yahoo! Cloud Serving Benchmark (YCSB). https://github.com/brianfrankcooper/YCSB. Accessed: 2019-06-10.Google Scholar
- Zipfian's Law. https://en.wikipedia.org/wiki/Zipf%27s_law. Accessed: 2019-06-10.Google Scholar
- Berk Atikoglu, Yuehai Xu, Eitan Frachtenberg, Song Jiang, and Mike Paleczny. 2012. Workload Analysis of a Large-scale Key-value Store. In Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '12). ACM, New York, NY, USA, 53--64. https://doi.org/10.1145/2254756.2254766Google ScholarDigital Library
- Hjortur Bjornsson, Gregory Chockler, Trausti Saemundsson, and Ymir Vigfusson. 2013. Dynamic performance profiling of cloud caches. In Proceedings of the 4th annual Symposium on Cloud Computing. ACM, Seattle, WA, USA, 1--14.Google ScholarDigital Library
- Aaron Blankstein, Siddhartha Sen, and Michael J. Freedman. 2017. Hyperbolic Caching: Flexible Caching for Web Applications. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 499--511. https://www.usenix.org/conference/atc17/technical-sessions/presentation/blanksteinGoogle ScholarDigital Library
- Burton H. Bloom. 1970. Space/Time Trade-offs in Hash Coding with Allowable Errors. Commun. ACM 13, 7 (July 1970), 422--426. https://doi.org/10.1145/362686.362692Google ScholarDigital Library
- Jacob Brock, Chencheng Ye, Ding Chen, Yechen Li, Xiaolin Wang, and Yingwei Luo. 2015. Optimal Cache Partition-Sharing. In International Conference on Parallel Processing, ICPP'15. 749--758.Google Scholar
- Daniel Byrne, Nilufer Onder, and Zhenlin Wang. 2018. mPart: Miss-ratio Curve Guided Partitioning in Key-value Stores. In Proceedings of the 2018 ACM SIGPLAN International Symposium on Memory Management (ISMM 2018). ACM, New York, NY, USA, 84--95. https://doi.org/10.1145/3210563.3210571Google ScholarDigital Library
- Pei Cao and Sandy Irani. 1997. Cost-aware WWW Proxy Caching Algorithms. In Proceedings of the USENIX Symposium on Internet Technologies and Systems (USITS'97). USENIX Association, Berkeley, CA, USA, 18--18. http://dl.acm.org/citation.cfm?id=1267279.1267297Google Scholar
- Calin Cascaval, Evelyn Duesterwald, Peter F. Sweeney, and Robert W. Wisniewski. 2005. Multiple Page Size Modeling and Optimization. 339--349.Google Scholar
- Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh, and Sachin Katti. 2015. Dynacache: Dynamic Cloud Caching. In 7th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 15). USENIX Association, Santa Clara, CA. https://www.usenix.org/conference/hotcloud15/workshop-program/presentation/cidonGoogle ScholarDigital Library
- Asaf Cidon, Assaf Eisenman, Mohammad Alizadeh, and Sachin Katti. 2016. Cliffhanger: Scaling Performance Cliffs in Web Memory Caches. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16). USENIX Association, Santa Clara, CA, 379--392. https://www.usenix.org/conference/nsdi16/technical-sessions/presentation/cidonGoogle ScholarDigital Library
- Asaf Cidon, Daniel Rushton, Stephen M. Rumble, and Ryan Stutsman. 2017. Memshare: a Dynamic Multi-tenant Key-value Cache. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 321--334. https://www.usenix.org/conference/atc17/technical-sessions/presentation/cidonGoogle ScholarDigital Library
- Xiameng Hu, Xiaolin Wang, Yechen Li, Lan Zhou, Yingwei Luo, Chen Ding, Song Jiang, and Zhenlin Wang. 2015. LAMA: Optimized locality-aware memory allocation for key-value cache. In Proceedings of USENIX ATC'15. 57--70.Google Scholar
- X. Hu, X. Wang, L. Zhou, Y. Luo, C. Ding, S. Jiang, and Z. Wang. 2017. Optimizing Locality-Aware Memory Management of Key-Value Caches. IEEE Trans. Comput. 66, 5 (May 2017), 862--875. https://doi.org/10.1109/TC.2016.2618920Google ScholarDigital Library
- Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Chen Ding, and Zhenlin Wang. 2016. Kinetic Modeling of Data Eviction in Cache. In 2016 USENIX Annual Technical Conference (USENIX ATC 16).Google ScholarDigital Library
- Xiameng Hu, Xiaolin Wang, Lan Zhou, Yingwei Luo, Zhenlin Wang, Chen Ding, and Chencheng Ye. 2018. Fast Miss Ratio Curve Modeling for Storage Cache. ACM Trans. Storage 14, 2, Article 12 (April 2018), 34 pages. https://doi.org/10.1145/3185751Google ScholarDigital Library
- Yul H. Kim, Mark D. Hill, and David A. Wood. 1991. Implementing Stack Simulation for Highly-Associative Memories. Acm Sigmetrics Performance Evaluation Review 19, 1, 212--213.Google ScholarDigital Library
- Conglong Li and Alan L. Cox. 2015. GD-Wheel: A Cost-aware Replacement Policy for Key-value Stores. In Proceedings of the Tenth European Conference on Computer Systems (EuroSys '15). ACM, New York, NY, USA, Article 5, 15 pages. ttps://doi.org/10.1145/2741948.2741956Google Scholar
- R. L. Mattson, J. Gecsei, D. R. Slutz, and I. L. Traiger. 1970. Evaluation Techniques for Storage Hierarchies. IBM Syst. J. 9, 2 (June 1970), 78--117. https://doi.org/10.1147/sj.92.0078Google ScholarDigital Library
- Qingpeng Niu, James Dinan, Qingda Lu, and P. Sadayappan. 2012. PARDA: A Fast Parallel Reuse Distance Analysis Algorithm. In Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium (IPDPS '12). IEEE Computer Society, Washington, DC, USA, 1284--1294. https://doi.org/10.1109/IPDPS.2012.117Google Scholar
- J. Ou, M. Patton, M. D. Moore, Y. Xu, and S. Jiang. 2015. A Penalty Aware Memory Allocation Scheme for Key-Value Cache. In 2015 44th International Conference on Parallel Processing. 530--539. https://doi.org/10.1109/ICPP.2015.62Google ScholarDigital Library
- Cheng Pan, Xiameng Hu, Lan Zhou, Yingwei Luo, Xiaolin Wang, and Zhenlin Wang. 2018. PACE: Penalty Aware Cache Modeling with Enhanced AET. In Proceedings of the 9th Asia-Pacific Workshop on Systems (APSys '18). ACM, New York, NY, USA, Article 19, 8 pages. https://doi.org/10.1145/3265723.3265736Google ScholarDigital Library
- J. Shim, P. Scheuermann, and R. Vingralek. 1999. Proxy cache algorithms: Design, implementation, and performance. Knowledge Data Engineering IEEE Transactions on 11, 4 (1999), 549--562.Google ScholarDigital Library
- G Edward Suh, Srinivas Devadas, and Larry Rudolph. 2001. Analytical cache models with applications to cache partitioning. In Proceedings of the 15th international conference on Supercomputing, ICS'01. 1--12.Google ScholarDigital Library
- Carl Waldspurger, Trausti Saemundsson, Irfan Ahmad, and Nohhyun Park. 2017. Cache Modeling and Optimization using Miniature Simulations. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 487--498. https://www.usenix.org/conference/atc17/technical-sessions/presentation/waldspurgerGoogle ScholarDigital Library
- Carl A Waldspurger, Nohhyun Park, Alexander Garthwaite, and Irfan Ahmad. 2015. Efficient MRC construction with SHARDS. In 13th USENIX Conference on File and Storage Technologies (FAST 15). USENIX Association, 95--110.Google ScholarDigital Library
- Jake Wires, Stephen Ingram, Zachary Drudi, Nicholas J. A. Harvey, and Andrew Warfield. 2014. Characterizing Storage Workloads with Counter Stacks. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, Broomfield, CO, 335--349. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/wiresGoogle ScholarDigital Library
- Chencheng Ye, Jacob Brock, Ding Chen, and Hai Jin. 2017. Rochester Elastic Cache Utility (RECU): Unequal Cache Sharing is Good Economics. International Journal of Parallel Programming 45, 1 (2017), 30--44.Google ScholarDigital Library
- Neal E. Young. 1991. Competitive Paging and Dual-Guided On-Line Weighted Caching and Matching Algorithms. In PhD thesis, Princeton University.Google Scholar
- Xiao Zhang, Sandhya Dwarkadas, and Kai Shen. 2009. Towards practical page coloring-based multicore cache management. In Proceedings of the 4th ACM European conference on Computer systems. ACM, 89--102.Google ScholarDigital Library
- Pin Zhou, Vivek Pandey, Jagadeesan Sundaresan, Anand Raghuraman, Yuanyuan Zhou, and Sanjeev Kumar. 2004. Dynamic Tracking of Page Miss Ratio Curve for Memory Management. In Proceedings of the 11th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XI). ACM, New York, NY, USA, 177--188. https://doi.org/10.1145/1024393.1024415Google ScholarDigital Library
Index Terms
pRedis: Penalty and Locality Aware Memory Allocation in Redis
Recommendations
Dynamically Configuring LRU Replacement Policy in Redis
MEMSYS '20: Proceedings of the International Symposium on Memory SystemsTo reduce the latency of accessing backend servers, today’s web services usually adopt in-memory key-value stores in the front end which cache the frequently accessed objects. Memcached and Redis are two most popular key-value cache systems. Due to the ...
Penalty- and Locality-aware Memory Allocation in Redis Using Enhanced AET
Due to large data volume and low latency requirements of modern web services, the use of an in-memory key-value (KV) cache often becomes an inevitable choice (e.g., Redis and Memcached). The in-memory cache holds hot data, reduces request latency, and ...
Set-associative cache simulation using generalized binomial trees
Set-associative caches are widely used in CPU memory hierarchies, I/O subsystems, and file systems to reduce average access times. This article proposes an efficient simulation technique for simulating a group of set-associative caches in a single pass ...
Comments