Skip to main content
SpringerLink
Log in

A feedback-based adaptive data migration method for hybrid storage VOD caching systems

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Nowadays Video-On-Demand (VOD) caching systems are often equipped with hybrid storage devices, which have been designed to combine the high read speed of Solid State Disks (SSDs) and the large capacity of Hard Disk Drives (HDDs). However, the number of erase cycles of SSDs is limited. So it is important to control the write load of SSDs in real applications. This paper proposes a Feedback-based Adaptive Data Migration (FADM) method, which can utilize the real-time feedback of the write load of SSDs to adjust the rule of moving data between HDDs and SSDs. More specifically, a video in HDDs is allowed to be moved into SSDs when its popularity is higher than that of the least popular video in SSDs by a threshold. This threshold is adaptively adjusted according to the feedback of the write load of SSDs. With FADM, the desired lifetime of SSDs can be well guaranteed even under various user behaviors while good read performance can be maintained. Simulations are done to demonstrate the effectiveness of FADM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Some wear leveling operations are done to well balance the write loads of different parts of a SSD [15–17]. So we just care about the total write load of SSDs without worrying about which part of a SSD a video should be written in.

  2. As the system does not know anything about the real write load at the beginning, the initial value δ[0] = 0 is chosen to place the least restrictive writing constraint on SSDs and make the best use of SSDs. Other initial values can also work for δ[0] if some a priori knowledge regarding the real write load is available.

  3. In reality, videos may not have the same length. Then we can first determine a reasonable segment size and partition videos with that given segment size. Of course, different videos may have different numbers of segments. Fortunately, all our approaches are still applicable because the concerned object in the caching procedure is segment, rather than video. Under that situation, m 1 denotes the number of segments of the longest video. The benefit of the equal length assumption is that we can know the total number of segments is equal to the multiplication of m 1 and the total number of videos.

  4. Nowadays a 100GB SSD and a 1TB HDD have comparable price. So we set that SSDs contribute 10 % of the total cache size. Of course, other value of C s can also work.

  5. In that case, we repeat this random choice until a qualified r 2 is obtained.

References

  1. Abhari A, Soraya M (2010) Workload generation for youtube. Multimed Tools and Appl 46(1):91–118

    Article  Google Scholar 

  2. Breslau L, Cao P, Fan L, Phillips G, Shenker S (1999) Web caching and zipf-like distributions: Evidence and implications. In: IEEE Eighteenth Annual Joint Conference on Computer and Communications Societies., pp. 126-134

  3. Chen T (2007) Obtaining the optimal cache document replacement policy for the caching system of an EC website. Eur J Oper Res 181(2):828–841

    Article  MATH  Google Scholar 

  4. Chen F, Koufaty DA, Zhang X (2011) Hystor: making the best use of solid state drives in high performance storage systems. In: Proceedings of the international conference on Supercomputing (ICS), pp. 22–32

  5. Cheong SK, Jeong JJ, Jeong YW, Ko DS, Lee YH (2011) Research on the I/O Performance Advancement of a Low Speed HDD Using DDR-SSD. In: Proceedings of the 6th International conference on Future Information Technology, pp 508–513

  6. Choi J, Reaz A, Mukherjee B (2012) A survey of user behavior in VoD service and bandwidth-saving multicast streaming schemes. IEEE Communications Surv and Tutorials 14(1):156–169

    Article  Google Scholar 

  7. Dorf R, Bishop R (2007) Modern control systems , 11th Edition. Prentice Hall, NJ

    Google Scholar 

  8. Dyaberi JM (2011) Networking and storage support for video-on-demand data delivery. Purdue University, West Lafayette, Indiana, USA

  9. Intel Corporation (1998) Understanding the flash translation layer (FTL) specification. Technical Report AP-684

  10. Kang JU, JoH, Kim JS, Lee J (2006) A superblock-based flash translation layer for NAND flash memory. In: Proceedings of the 6th ACM IEEE International conference on Embedded software. pp. 161–170

  11. Kim J , Kim JM, Noh SH , Min SL , Cho Y (2002) A space-efficient flash translation layer for compact flash systems. IEEE Trans on Consum Electron 48(2):366–375

    Article  Google Scholar 

  12. Kim YJ, Lee SJ, Zhang K, Kim J (2007) I/O performance optimization techniques for hybrid hard disk-based mobile consumer devices. IEEE Trans Consum Electron 53(4):1469–1476

    Article  Google Scholar 

  13. Koltsidas I, Viglas SD (2008) Flashing up the storage layer. In: Proceedings of VLDB, pp. 514–525

  14. Li R, Li C, Xiao W (2012) An efficient SSD-based hybrid storage architecture for large-scale search engines. In: Proceedings of the 41th International Conference on Parallel Processing, pp. 450–459

  15. Lin L, Zhu Y, Yue J, Cai Z, Segee B (2011) Hot random off-loading: A hybrid storage system with dynamic data migration. In: Proceedings of 19th Annual IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems, pp. 318–325

  16. Narayanan D, Thereska E, Donnelly A, Elnikety S, Rowstron A (2009) Migrating server storage to SSDs: analysis of tradeoffs. In: Proceedings of the 4th ACM European Conference on computer systems, pp. 145–158

  17. Reisslein M, Hartanto F, Ross KW (2002) Interactive video streaming with proxy servers. Information Science, special issue on Interactive virtual environments and distance education 140(1-2):3–31

    MATH  Google Scholar 

  18. Robinson JT, Devarakonda MV (1990) Data cache management using frequency-based replacement. In: Proceedings of the 1990 ACM SIGMETRICS on the Measurement and Modeling of Computer Systems, pp. 132-142

  19. Soundararajan G, Prabhakaran V, Balakrishnan M, Wobber T (2010) Extending SSD lifetimeswith disk- Based write caches. In: Proceedings of the 8th USENIX Conference on File and Storage Technologies (FAST 10), pp. 101–114

  20. LFU-K: An effective buffer management replacement algorithm (2004)

  21. LRU-based algorithms for web cache replacement (2000)

  22. Wolf JL, Yu PS, Shachnai H (1997) Disk load balancing for video-on-demand systems. Multimed Syst 5(6):358–370

    Article  Google Scholar 

  23. Wu KL, Yu PS, Wolf JL (2004) Segmentation of multimedia streams for proxy caching. IEEE Trans Multimed 6(5):770–780

    Article  Google Scholar 

  24. Wu X, Reddy AN (2009) Managing storage space in a flash and disk hybrid storage system. In: Proceedings of IEEE MASCOTS, pp. 583–587

  25. Wu X, Reddy AN (2010) Exploiting concurrency to improve latency and throughput in a hybrid storage system. In: Proceedings of IEEE MASCOTS, pp. 14–23

  26. Wu X, Reddy AN (2012) Data organization in a hybrid storage system. In: Proceedings of International Conference on Computing, Networking and Communications (ICNC), pp. 583–587

  27. Yoon UK, Kim HJ (2009) Improving Hybrid Flash-Disk Storage by Incorporating Sequential Pattern Mining into Fully-Associative Sector Translation. In: Proceedings of International Conferenceon Computational Science and Engineering, pp 189–196

  28. Yoon UK, Kim HJ, Chang JY (2010) Intelligent Data Prefetching for Hybrid Flash-Disk Storage Using Sequential Pattern Mining Technique. In: Proceedings of IEEE/ACIS 9th International Conference on Computer and Information Science, pp 280–285

  29. Yu J, Chou CT, Yang Z, Du X,Wang T (2006) A dynamic caching algorithm based on internal popularity distribution of streaming media. Multimed Syst 12(2):135–149

    Article  Google Scholar 

Download references

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China (No. 61273112), the Fundamental Research Funds for the Central Universities and the 973 Program (No. 2013CB733100).

Author information

Authors and Affiliations

  1. Department of Automation, University of Science and Technology of China, Hefei, 230027, China

    Qiang Ling, Lixiang Xu, Jinfeng Yan, Yicheng Zhang & Feng Li

Authors
  1. Qiang Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lixiang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinfeng Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yicheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Feng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiang Ling.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ling, Q., Xu, L., Yan, J. et al. A feedback-based adaptive data migration method for hybrid storage VOD caching systems. Multimed Tools Appl 75, 165–180 (2016). https://doi.org/10.1007/s11042-014-2281-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2281-y

Keywords

Search

Navigation