ABSTRACT
Content streaming is the dominant application in today's Internet, which is typically distributed via content delivery networks (CDNs). CDNs usually use caching as a means to reduce user access latency so as to enable faster content downloads. Typical analysis of caching systems either focuses on content admission, which decides whether to cache a content, or content eviction to decide which content to evict when the cache is full. This paper instead proposes a novel framework that can simultaneously learn both content admission and content eviction for caching in CDNs. To attain this goal, we first put forward a lightweight architecture for content next request time prediction. We then leverage reinforcement learning (RL) along with the prediction to learn the time-varying content popularities for content admission, and develop a simple threshold-based model for content eviction. We call this new algorithm RL-Bélády (RLB). In addition, we address several key challenges to design learning-based caching algorithms, including how to guarantee lightweight training and prediction with both content eviction and admission in consideration, limit memory overhead, reduce randomness and improve robustness in RL stochastic optimization. Our evaluation results using $3$ production CDN datasets show that RLB can consistently outperform state-of-the-art methods with dramatically reduced running time and modest overhead.
Supplemental Material
- Martin Arlitt, Ludmila Cherkasova, John Dilley, Rich Friedrich, and Tai Jin. 2000. Evaluating Content Management Techniques for Web Proxy Caches. ACM SIGMETRICS Performance Evaluation Review 27, 4 (2000), 3--11.Google ScholarDigital Library
- Laszlo A. Bélády. 1966. A Study of Replacement Algorithms for A Virtual-Storage Computer. IBM Systems journal 5, 2 (1966), 78--101.Google ScholarDigital Library
- Daniel S Berger. 2018. Towards Lightweight and Robust Machine Learning for CDN Caching. In Proc. of ACM HotNets.Google ScholarDigital Library
- Daniel S Berger, Ramesh K Sitaraman, and Mor Harchol-Balter. 2017. AdaptSize: Orchestrating the Hot Object Memory Cache in a Content Delivery Network. In Proc. of USENIX NSDI.Google Scholar
- Bilibili. [n.d.]. https://www.bilibili.com.Google Scholar
- Meeyoung Cha, Haewoon Kwak, Pablo Rodriguez, Yong-Yeol Ahn, and Sue Moon. 2009. Analyzing the Video Popularity Characteristics of Large-Scale User Generated Content Systems. IEEE/ACM Transactions on networking 17, 5 (2009), 1357--1370.Google Scholar
- Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proc. of ACM SIGKDD.Google ScholarDigital Library
- Ludmila Cherkasova. 1998. Improving WWW Proxies Performance with GreedyDual-Size-Frequency Caching policy. Hewlett-Packard Laboratories.Google Scholar
- VNI Cisco. 2015. Cisco Visual Networking Index: Forecast and Methodology 2014--2019 White Paper. Cisco, Tech. Rep (2015).Google Scholar
- Gil Einziger, Roy Friedman, and Ben Manes. 2017. TinyLFU: A Highly Efficient Cache Admission Policy. ACM Transactions on Storage (ToS) 13, 4 (2017), 1--31.Google ScholarDigital Library
- Qilin Fan, Jian Li, Xiuhua Li, Qiang He, Shu Fu, and Sen Wang. 2020. PA-Cache: Learning-based Popularity-Aware Content Caching in Edge Networks. arXiv preprint arXiv:2002.08805 (2020).Google Scholar
- Vladyslav Fedchenko, Giovanni Neglia, and Bruno Ribeiro. 2019. Feedforward Neural Networks for Caching: Enough or Too Much? ACM SIGMETRICS Performance Evaluation Review 46, 3 (2019), 139--142.Google ScholarDigital Library
- N Choungmo Fofack, Philippe Nain, Giovanni Neglia, and Don Towsley. 2012. Analysis of TTL-based Cache Networks. In Proc. of IEEE VALUETOOLS.Google Scholar
- Jerome H Friedman. 2001. Greedy Function Approximation: A Gradient Boosting Machine. Annals of statistics (2001), 1189--1232.Google Scholar
- Yu Guan, Xinggong Zhang, and Zongming Guo. 2019. CACA: Learning-based Content-Aware Cache Admission for Video Content in Edge Caching. In Proc. of ACM Multimedia.Google ScholarDigital Library
- Syed Hasan, Sergey Gorinsky, Constantine Dovrolis, and Ramesh K Sitaraman. 2014. Trade-Offs in Optimizing the Cache Deployments of CDNs. In Proc. of IEEE INFOCOM.Google ScholarCross Ref
- W. Jiang, S. Ioannidis, L. Massoulié, and F. Picconi. 2012. Orchestrating Massively Distributed CDNs. In Proc. of ACM CoNEXT.Google Scholar
- Vadim Kirilin, Aditya Sundarrajan, Sergey Gorinsky, and Ramesh K Sitaraman. 2019. RL-Cache: Learning-based Cache Admission for Content Delivery. In Proc. of Workshop on Network Meets AI & ML.Google ScholarDigital Library
- Jian Li, Srinivas Shakkottai, John CS Lui, and Vijay Subramanian. 2018. Accurate Learning or Fast Mixing? Dynamic Adaptability of Caching Algorithms. IEEE Journal on Selected Areas in Communications 36, 6 (2018), 1314--1330.Google ScholarDigital Library
- Ge Ma, Zhi Wang, Miao Zhang, Jiahui Ye, Minghua Chen, and Wenwu Zhu. 2017. Understanding Performance of Edge Content Caching for Mobile Video Streaming. IEEE Journal on Selected Areas in Communications 35, 5 (2017), 1076-- 1089.Google ScholarDigital Library
- Bruce M Maggs and Ramesh K Sitaraman. 2015. Algorithmic Nuggets in Content Delivery. ACM SIGCOMM Computer Communication Review 45, 3 (2015), 52--66.Google ScholarDigital Library
- Nimrod Megiddo and Dharmendra S Modha. 2003. ARC: A Self-Tuning, Low Overhead Replacement Cache.. In Proc. of USENIX FAST.Google Scholar
- Kathlene Morales and Byeong Kil Lee. 2012. Fixed Segmented LRU Cache Replacement Scheme with Selective Caching. In Proc. of IEEE IPCCC.Google ScholarCross Ref
- Arvind Narayanan, Saurabh Verma, Eman Ramadan, Pariya Babaie, and ZhiLi Zhang. 2018. Deepcache: A Deep Learning Based Framework for Content Caching. In Proc. of Workshop on Network Meets AI & ML.Google ScholarDigital Library
- Elizabeth J O-neil, Patrick E O-neil, and Gerhard Weikum. 1993. The LRU-K Page Replacement Algorithm for Database Disk Buffering. Acm Sigmod Record 22, 2 (1993), 297--306.Google ScholarDigital Library
- Nitish K Panigrahy, Jian Li, and Don Towsley. 2017. Hit Rate vs. Hit Probability Based Cache Utility Maximization. ACM SIGMETRICS Performance Evaluation Review 45, 2 (2017), 21--23.Google ScholarDigital Library
- Nitish K Panigrahy, Jian Li, Don Towsley, and Christopher V Hollot. 2020. Network Cache Design under Stationary Requests: Exact Analysis and Poisson Approximation. Computer Networks (2020), 107379.Google Scholar
- Nitish K Panigrahy, Jian Li, Faheem Zafari, Don Towsley, and Paul Yu. 2017. Optimizing Timer-based Policies for General Cache Networks. arXiv preprint arXiv:1711.03941 (2017).Google Scholar
- Georgios S Paschos, George Iosifidis, Meixia Tao, Don Towsley, and Giuseppe Caire. 2018. The Role of Caching in Future Communication Systems and Networks. IEEE Journal on Selected Areas in Communications 36, 6 (2018), 1111--1125.Google ScholarDigital Library
- Ketan Shah, Anirban Mitra, and Dhruv Matani. 2010. An O(1) Algorithm for Implementing the LFU Cache Eviction Scheme. no 1 (2010), 1--8.Google Scholar
- Zhenyu Song, Daniel S Berger, Kai Li, and Wyatt Lloyd. 2020. Learning Relaxed Belady for Content Distribution Network Caching. In Proc. of USENIX NSDI.Google Scholar
- Kalika Suksomboon, Saran Tarnoi, Yusheng Ji, Michihiro Koibuchi, Kensuke Fukuda, Shunji Abe, Nakamura Motonori, Michihiro Aoki, Shigeo Urushidani, and Shigeki Yamada. 2013. PopCache: Cache More or Less Based on Content Popularity for Information-Centric Networking. In Proc. of IEEE Conference on Local Computer Networks.Google ScholarCross Ref
- Michael Zink, Kyoungwon Suh, Yu Gu, and Jim Kurose. 2008. Watch Global, Cache Local: YouTube Network Traffic at A Campus Network: Measurements and Implications. In Multimedia Computing and Networking 2008, Vol. 6818. 681805.Google ScholarCross Ref
Index Terms
- RL-Bélády: A Unified Learning Framework for Content Caching
Recommendations
CACA: Learning-based Content-aware Cache Admission for Video Content in Edge Caching
MM '19: Proceedings of the 27th ACM International Conference on MultimediaIn the last decades, network caches (Content Distribution Network, CDN) have been widely deployed in video delivery system. As cache has been pushed to network edge as far as possible, small cache size and irregular request pattern make it a great ...
RL-Cache: Learning-Based Cache Admission for Content Delivery
NetAI'19: Proceedings of the 2019 Workshop on Network Meets AI & MLContent delivery networks (CDNs) distribute much of the Internet content by caching and serving the objects requested by users. A major goal of a CDN is to maximize the hit rates of its caches, thereby enabling faster content downloads to the users. ...
HEC: improving endurance of high performance flash-based cache devices
SYSTOR '13: Proceedings of the 6th International Systems and Storage ConferenceFlash memory is widely used for its fast random I/O access performance in a gamut of enterprise storage applications. However, due to the limited endurance and asymmetric write performance of flash memory, minimizing writes to a flash device is critical ...
Comments