Skip to main content
SpringerLink
Log in

A novel POMDP-based server RAM caching algorithm for VoD systems

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In contrast to other services on the Internet, streaming media service needs to fetch data from local disks more frequently, since it always lasts long and the bit rate is quite high. In addition, because of the much slower reading/writing speed of disk than random access memory (RAM), adopting advisable RAM caching policy can efficiently reduce disk I/O. In this paper, we study the problem of reducing disk I/O by using a novel approach. We first provide a new popularity estimate algorithm. Then a formal optimization problem about average disk I/O is presented, and a suboptimal caching algorithm for a special case of the problem is given. Furthermore, a partially observable Markov decision process (POMDP) model is constructed for the caching problem. Based on the model, popularity is taken advantage of to predict clients’ randomized behaviors, data replacing decisions are made when the defined observations occur, and the impact of caching actions on disk performance for future infinite steps is assessed. The method of event-based optimization is applied in search of the optimal stochastic policy. Disk I/O, as the long-run average performance measure, is optimized by applying the policy-gradient algorithm. The simulation results illustrate that data required by clients could be better predicted and lower disk I/O could be achieved by using the model proposed in this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Acharya S, Smith BC, Parnes P (1999) Characterizing user access to videos on the world wide web Electronic Imaging, International Society for Optics and Photonics, pp 130–141

  2. Aggarwal G, Kumar G, Sethia D (2012) Dynamic prefix caching of videos with lazy update Proceedings of the second international conference on Computational Science, Engineering and Information Technology. ACM, pp 689–693

  3. Baxter J, Bartlett PL (2001) Infinite-horizon policy-gradient estimation. J Artif Intell Res 15:319–350

    MathSciNet  MATH  Google Scholar 

  4. Bhat UN (2015) An introduction to queueing theory: modeling and analysis in applications, Birkhäuser

  5. Borst S, Gupta V, Walid A (2009) Self-organizing algorithms for cache cooperation in content distribution networks. Bell Labs Tech J 14(3):113–125

    Article  Google Scholar 

  6. Cao XR, Zhang J (2008) Event-based optimization of markov systems. IEEE Trans Autom Control 53(4):1076–1082

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen S, Shen B, Wee S, Zhang X (2006) Segment-based streaming media proxy: modeling and optimization. IEEE Trans Multimedia 8(2):243–256

    Article  Google Scholar 

  8. Chesire M, Wolman A, Voelker GM, Levy HM (2001) Measurement and analysis of a streaming media workload USITS, vol 1, pp 1–1

  9. Cormen T, Leiserson C, Rivest R (1990) Introduction to algorithms. MIT Press, Cambridge

    MATH  Google Scholar 

  10. Dan A, Sitaram D (1994) Buffer management policy for an on-demand video server. Citeseer, Yorktown Heights

  11. Dan A, Sitaram D (1996) Generalized interval caching policy for mixed interactive and long video workloads Electronic Imaging: Science & Technology, International Society for Optics and Photonics, pp 344–351

  12. Eberhard M, Szkaliczki T, Hellwagner H, Szobonya L, Timmerer C (2010) Knapsack problem-based piece-picking algorithms for layered content in peer-to-peer networks. ACM

  13. Gomaa H, Messier GG, Williamson C, Davies R (2013) Estimating instantaneous cache hit ratio using markov chain analysis. IEEE/ACM Trans Networking 21(5):1472–1483

    Article  Google Scholar 

  14. Gramatikov S, Jaureguizar F (2016) Modelling and analysis of non-cooperative peer-assisted vod streaming in managed networks. Multimedia Tools and Applications 75(8):4321–4348

    Article  Google Scholar 

  15. He Y, Shen G, Xiong Y, Guan L (2009) Optimal prefetching scheme in p2p vod applications with guided seeks. IEEE Trans Multimedia 11(1):138–151

    Article  Google Scholar 

  16. Hsu SP, Chuang DM, Arapostathis A (2006) On the existence of stationary optimal policies for partially observed mdps under the long-run average cost criterion. Syst Control Lett 55(2):165–173

    Article  MathSciNet  MATH  Google Scholar 

  17. Hwang KS, Jiang WC, Chen YJ (2015) Model learning and knowledge sharing for a multiagent system with dyna-q learning. IEEE transactions on cybernetics 45(5):978–990

    Article  Google Scholar 

  18. Lee D, Choi J, Kim JH, Noh SH, Min SL, Cho Y, Kim CS (2001) Lrfu: a spectrum of policies that subsumes the least recently used and least frequently used policies. IEEE Trans Comput 50(12):1352–1361

    Article  MathSciNet  Google Scholar 

  19. Li Y, Yin B, Xi H (2008) Partially observable markov decision processes and performance sensitivity analysis. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 38(6):1645–1651

    Article  Google Scholar 

  20. Lim H, Du DH (2001) Protocol considerations for video prefix-caching proxy in wide area networks. Electron Lett 37(6):1

    Article  Google Scholar 

  21. Ling Q, Xu L, Yan J, Zhang Y (2015) An adaptive caching algorithm suitable for time-varying user accesses in vod systems. Multimedia Tools and Applications 74 (24):11,117–11,137

    Article  Google Scholar 

  22. Lu X, Yin B, Zhang X, Cao J, Kang Y (2016) Event-based optimization for admission control in distributed service system. Telecommun Syst 62(3):553–567

    Article  Google Scholar 

  23. Luo JG, Zhang Q, Tang Y, Yang SQ (2009) A trace-driven approach to evaluate the scalability of p2p-based video-on-demand service. IEEE Trans Parallel Distrib Syst 20(1):59–70

    Article  Google Scholar 

  24. Megiddo N, Modha DS (2004) Outperforming lru with an adaptive replacement cache algorithm. Computer 37(4):58–65

    Article  Google Scholar 

  25. Oh HR, Song H (2007) Metafile-based scalable caching and dynamic replacing algorithms for multiple videos over quality-of-service networks. IEEE Trans Multimedia 9(7):1535–1542

    Article  Google Scholar 

  26. Psounis K, Prabhakar B (2002) Efficient randomized web-cache replacement schemes using samples from past eviction times. IEEE/ACM Trans Networking (TON) 10(4):441–455

    Article  Google Scholar 

  27. Puterman ML (2014) Markov decision processes: discrete stochastic dynamic programming, Wiley

  28. Rizzo L, Vicisano L (2000) Replacement policies for a proxy cache. IEEE/ACM Trans Networking (ToN) 8(2):158–170

    Article  Google Scholar 

  29. Sen S, Rexford J, Towsley D (1999) Proxy prefix caching for multimedia streams IEEE Proceedings of the INFOCOM’99. 18th Annual joint conference of the IEEE computer and communications societies, vol 3. IEEE, pp 1310–1319

  30. Senda K, Hattori S, Hishinuma T, Kohda T (2014) Acceleration of reinforcement learning by policy evaluation using nonstationary iterative method. IEEE trans cybernetics 44(12):2696–2705

    Article  Google Scholar 

  31. Sprangers O, Babuṡka R, Nageshrao SP, Lopes GA (2015) Reinforcement learning for port-hamiltonian systems. IEEE trans cybernetics 45(5):1017–1027

    Article  Google Scholar 

  32. Tewari R, Vin HM, Dan A, Sitaram D (1997) Resource-based caching for web servers Photonics West’98 Electronic Imaging, International Society for Optics and Photonics, pp 191–204

    Google Scholar 

  33. Wilson L, Zipf GK (1949) Human behavior and the principle of least effort

  34. Wu KL, Yu PS, Wolf JL (2004) Segmentation of multimedia streams for proxy caching. IEEE Trans Multimedia 6(5):770–780

    Article  Google Scholar 

  35. Xie H, Gao L, Zhang L, Zhang Z, Yang M (2009) An efficient caching mechanism for video-on-demand service over peer-to-peer network International Conference on Scalable computing and communications; 8th international conference on embedded computing, 2009. SCALCOM-EMBEDDEDCOM’09. IEEE, pp 251–256

  36. Yin B, Lu S, Guo D (2011) Analysis of admission control in p2p-based media delivery network based on pomdp. International Journal of Innovative Computing Information and Control 7(7B):4411–4422

    Google Scholar 

  37. Zhou Y, Chiu DM, Lui JC (2011) A simple model for chunk-scheduling strategies in p2p streaming. IEEE/ACM Trans Networking 19(1):42–54

    Article  Google Scholar 

  38. Zhou Y, Fu TZ, Chiu DM (2015) A unifying model and analysis of p2p vod replication and scheduling. IEEE/ACM Trans Networking (TON) 23(4):1163–1175

    Article  Google Scholar 

  39. Zink M, Suh K, Gu Y, Kurose J (2009) Characteristics of youtube network traffic at a campus network–measurements, models, and implications. Comput Netw 53 (4):501–514

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported in part by the National Natural Science Foundation of China under grant Nos. 61174124, 61233003, 61503358, 61673361.

Author information

Authors and Affiliations

  1. Department of Automation, University of Science and Technology of China, Hefei, 230027, China

    Baoqun Yin, Jie Cao, Yu Kang & Xiaofeng Jiang

  2. School of Management, Hefei University of Technology, Hefei, 230009, China

    Xiaonong Lu

Authors
  1. Baoqun Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaonong Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaofeng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baoqun Yin.

Additional information

This work is supported in part by the National Natural Science Foundation of China under grant Nos. 61174124, 61233003,61304048 in part by Research Fund for the Doctoral Program of Higher Education of China under grant No. 20123402110029 and in part by Natural Science Research Program of the Anhui High Education Bureau of China under grant No. KJ2012A286.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, B., Cao, J., Kang, Y. et al. A novel POMDP-based server RAM caching algorithm for VoD systems. Multimed Tools Appl 77, 13023–13045 (2018). https://doi.org/10.1007/s11042-017-4930-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4930-4

Keywords

Search

Navigation