Skip to main content
SpringerLink
Log in

CloudDMSS: robust Hadoop-based multimedia streaming service architecture for a cloud computing environment

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The delivery of scalable, rich multimedia applications and services on the Internet requires sophisticated technologies for transcoding, distributing, and streaming content. Cloud computing provides an infrastructure for such technologies, but specific challenges still remain in the areas of task management, load balancing, and fault tolerance. To address these issues, we propose a cloud-based distributed multimedia streaming service (CloudDMSS), which is designed to run on all major cloud computing services. CloudDMSS is highly adapted to the structure and policies of Hadoop, thus it has additional capacities for transcoding, task distribution, load balancing, and content replication and distribution. To satisfy the design requirements of our service architecture, we propose four important algorithms: content replication, system recovery for Hadoop distributed multimedia streaming, management for cloud multimedia management, and streaming resource-based connection (SRC) for streaming job distribution. To evaluate the proposed system, we conducted several different performance tests on a local testbed: transcoding, streaming job distribution using SRC, streaming service deployment and robustness to data node and task failures. In addition, we performed three different tests in an actual cloud computing environment, Cloudit 2.0: transcoding, streaming job distribution using SRC, and streaming service deployment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Dirk, W., Andreas, E., Alexandra, C.: An ecosystem for user-generated mobile services. J. Converg. 3(4), 43–48 (2012)

    Google Scholar 

  2. Barlas, G.: Cluster-based optimized parallel video transcoding. Parallel Comput. 38(4–5), 226–244 (2012)

    Article  Google Scholar 

  3. Ratakonda, K., Turaga, D.S., Lai, J.: QoS support for streaming media using a multimedia server cluster. In: IEEE GLOBECOM

  4. Guo, J., Chen, F., Bhuyan, L., Kumar, R.: A cluster-based active router architecture supporting video/audio stream transcoding service. In: Proceedings of Parallel and Distributed Processing Symposium (2003)

  5. Seo, D., Lee, J., Kim, Y., Choi, C., Jung, I.: Load distribution strategies in cluster-based transcoding servers for mobile clients. In: Proceedings of ICCSA, pp. 1156–1165 (2006)

  6. Strufe, T.: llmStream: efficient multimedia streaming in decentralized distributed systems. In: IFMIP, pp. 63–68 (2004)

  7. Kim, M., Han, S., Cui, Y., Lee, H., Jeong, C.: A Hadoop-based multimedia transcoding system for processing social media in the PaaS platform of SMCCSE. KSII Trans. Internet Inf. Syst. 6(11), 2827–2848 (2012)

    Google Scholar 

  8. Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandi, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6), 599–616 (2009)

    Article  Google Scholar 

  9. Vouk, M.A.: Cloud computing: issues, research, and implementations. In: ITI, pp. 31–40 (2008)

  10. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., Zaharia, M.: A view of cloud computing. Commun. ACM 53(4), 50–58 (2010)

    Article  Google Scholar 

  11. Grossman, R.L.: The case for cloud computing. IT Prof. 11(2), 23–27 (2009)

    Article  Google Scholar 

  12. Zhang, Q., Cheng, L., Boutaba, R.: Cloud computing: state of the art and research challenges. J. Internet Ser. A 1(1), 7–18 (2010)

    Article  Google Scholar 

  13. Pan, Y., Zhang, J.: Parallel programming on cloud computing platforms: challenges and solutions. J. Converg. 3(4), 23–28 (2012)

    MathSciNet  Google Scholar 

  14. Lei, Z.: Media transcoding for pervasive computing. In: ACM Conference on Multimedia Workshop, pp. 459–460 (2001)

  15. Lee, H.S., Lim, K.H., Kim, S.J.: A configuration scheme for connectivity-aware mobile P2P networks for efficient mobile cloud-based video streaming services. Clust. Comput. 1–12 (2013)

  16. Zhu, W., Luo, C., Wang, J., Li, S.: Multimedia cloud computing. IEEE Signal Process. Mag. 28(3), 59–69 (2011)

    Article  Google Scholar 

  17. Park, S., Jung, I.Y., Eom, H., Yeom, H.Y.: An analysis of replication enhancement for a high availability cluster. J. Inf. Process. Syst. 9(2), 205–216 (2013)

    Google Scholar 

  18. Apache Hadoop. http://hadoop.apache.org/. Accessed 5 May 2013

  19. Joey, J.: Introduction to Hadoop. http://i.dell.com/sites/content/business/solutions/whitepapers/en/Documents/hadoop-introduction.pdf. Accessed 2 Aug 2013

  20. HDFS architecture guide. http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html. Accessed 18 Aug 2013

  21. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoop distributed file system. In: MSST

  22. Ghemawat, S., Gobioff, H., Leung, S.T.: The Google file system. Oper. Syst. Rev. ACM 37(5), 29–43 (2003)

    Article  Google Scholar 

  23. Poellabauer, C., Schwan, K.: Energy-aware media transcoding in wireless systems. In: IEEE PerCom, pp. 135–144 (2004)

  24. Roy, S., Shen, B., Sundaram, V.: Application level hand off support for mobile media transcoding sessions. In: Workshop on Network and Operating Systems, pp. 95–104 (2002)

  25. Liao, X., Jin, H.: A new distributed storage scheme for cluster video server. J. Syst. Archit. 51(2), 79–94 (2005)

    Article  Google Scholar 

  26. Sambe, Y., Watanabe, S., Yu, D., Nakamura, T., Wakamiya, N.: High-speed distributed video transcoding for multiple rates and formats. IEICE Trans. Inf. Syst. E88–D(8), 1923–1931 (2005)

    Article  Google Scholar 

  27. Tian, Z., Xue, J., Hu, W., Xu, T., Zheng, N.: High performance cluster-based transcoder. In: Proceedings of ICCASM, pp. 248–252 (2010)

  28. Hui, W., Lin, C., Yang, Y.: MediaCloud: a new paradigm of multimedia computing. KSII Trans. Internet Inf. Syst. 6(4), 1153–1170 (2012)

  29. Luo, H., Egbert, A., Stahlhut, T.: QoS architecture for cloud-based media computing. In: ICSESS, pp. 769–772 (2012)

  30. Chang, S.Y., Lai, C.F., Huang, Y.H.: Dynamic adjustable multimedia streaming service architecture over cloud computing. Comput. Commun. 35(15), 1798–1808 (2012)

    Article  Google Scholar 

  31. Huang, Z., Mei, C., Li, L.E., Woo, T.: CloudStream: delivering high-quality streaming videos through a cloud-based SVC proxy. In: Proceedings of IEEE INFOCOM, pp. 201–205 (2011)

  32. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  33. Xuggler Java Library. http://www.xuggle.com/xuggler/index. Accessed 2 May 2013

  34. Kim, M., Cui, Y., Han, S., Lee, H.P.: Towards efficient design and implementation of a Hadoop-based distributed video transcoding system in cloud computing environment. J. Multimed. Ubiquitous Eng. 8(2), 213–224 (2013)

    Google Scholar 

  35. Piorkowski, A., Kempny, A., Hajduk, A., Strzelczyk, J.: Load balancing for heterogeneous web servers. CN, pp. 189–198 (2010)

  36. Swfupload. https://code.google.com/p/swfupload/. Accessed 10 Aug 2013

  37. NginX. http://nginx.org/. Accessed 10 Aug 2013

  38. Media Encoding Cluster Project. http://www.encodingcluster.com. Accessed 20 July 2013

  39. Cloudit 2.0. http://www.cloudit.co.kr/. Accessed 13 Jan 2014

  40. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Proceedings of ACM SOSP, pp. 164–177(2003)

Download references

Acknowledgments

This research was supported by the MSIP (Ministry of Science, ICT & Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (NIPA-2013-H0301-13-3006) supervised by the NIPA (National IT Industry Promotion Agency)

Author information

Authors and Affiliations

  1. Department of Internet and Multimedia Engineering, Konkuk University, Seoul, Korea

    Myoungjin Kim, Seungho Han, Yun Cui & Hanku Lee

  2. Technical Support Team, Innogrid Co., Ltd., Seoul, Korea

    Hogyeon Cho & Sungdae Hwang

Authors
  1. Myoungjin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seungho Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yun Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hanku Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hogyeon Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sungdae Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanku Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, M., Han, S., Cui, Y. et al. CloudDMSS: robust Hadoop-based multimedia streaming service architecture for a cloud computing environment. Cluster Comput 17, 605–628 (2014). https://doi.org/10.1007/s10586-014-0381-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-014-0381-0

Keywords

Search

Navigation