Abstract
Video surveillance applications need video data center to provide elastic virtual machine (VM) provisioning. However, the workloads of the VMs are hardly to be predicted for online video surveillance service. The unknown arrival workloads easily lead to workload skew among VMs. In this paper, we study how to balance the workload skew on online video surveillance system. First, we design the system framework for online surveillance service which consists of video capturing and analysis tasks. Second, we propose StreamTune, an online resource scheduling approach for workload balancing, to deal with irregular video analysis workload with the minimum number of VMs. We aim at timely balancing the workload skew on video analyzers without depending on any workload prediction method. Furthermore, we evaluate the performance of the proposed approach using a traffic surveillance application. The experimental results show that our approach is well adaptive to the variation of workload and achieves workload balance with less VMs.
Similar content being viewed by others
References
Hu H, Wen Y G, Chua T S, Li X L. Toward scalable systems for big data analytics: a technology tutorial. IEEE Access, 2014, 2: 652–687
Ma H D, Liu L, Zhou A F, Zhao D. On networking of Internet of things: explorations and challenges. IEEE Internet of Things Journal, 2016, 3(4): 441–452
Ma H D. Internet of things: objectives and scientific challenges. Journal of Computer Science and Technology, 2011, 26(6): 919–924
Zhu W W, Luo C, Wang J F, Li S P. Multimedia cloud computing. IEEE Signal Processing Magazine, 2011, 28(3): 59–69
Ahlgren B, Aranda P A, Chemouil P, Oueslati S, Correia L M, Karl H, Sollner M, Welin A. Content, connectivity and cloud: ingredients for the network of the future. IEEE Communication Magazine, 2011, 49(7): 62–70
Yang L, Cao J N, Yuan Y, Li T, Han A, Chan A. A framework for partitioning and execution of data stream applications in mobile cloud computing. ACM SIGMETRICS Performance Evaluation Review, 2013, 40(4): 23–32
Gao Y H, Ma H D, Zhang H T, Kong X Q, Wei W Y. Concurrency optimized task scheduling for workflows in cloud. In: Proceedings of the 6th IEEE International Conference on Cloud Computing. 2013, 709–716
Qian Z P, He Y, Su C Z, Wu Z J, Zhu H Y, Zhang T Z, Zhou L D, Yu Y, Zhang Z. Time Stream: reliable stream computation in the cloud. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 1–14
Zhao X M, Ma H D, Zhan H T, Tang Y, Kou Y. HVPI: extending Hadoop to support video analytic applications. In: Proceedings of the 8th IEEE International Conference on Cloud Computing. 2014, 789–796
Liu W, Mei T, Zhang Y D, Che C, Luo J B. Multi-task deep visualsemantic embedding for video thumbnail selection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3707–3715
Liu W, Zhang Y D, Tang S, Tang J H, Hong R, Li J T. Accurate estimation of human body orientation from RGB-D sensors. IEEE Transactions on Cybernetics, 2013, 43(5): 1442–1452
Liu W, Mei T, Zhang Y D. Instant mobile video search with layered audio-video indexing and progressive transmission. IEEE Transactions on Multimedia, 2014, 16(8): 2242–2255
Saini M, Wang X, Atrey P K, Kankanhalli M. Adaptive workload equalization in multi-camera surveillance systems. IEEE Transactions on Multimedia, 2012, 14(3): 555–562
Gao G Y, Zhang W W, Wen Y G, Wang Z, Zhu WW, Tan Y P. Cost optimal video transcoding in media cloud: insights from user viewing pattern. In: Proceedings of IEEE International Conference on Multimedia & Expo. 2014, 1–6
Ren S L, Van der Schaar M. Efficient resource provisioning and rate selection for stream mining in a community cloud. IEEE Transactions on Multimedia, 2013, 15(4): 723–734
Kwon Y C, Balazinska M, Rolia J. Skew-resistant parallel processing of feature-extracting scientific user-defined functions. In: Proceedings of the 1st ACM Symposium on Cloud Computing. 2010, 75–86
Pavlo A, Curino C, Zdonik S. Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems. In: Proceedings of ACM SIGMOD International Conference on Management of Data. 2012, 61–72
Ramakrishnan S R, Swart G, Urmanov A. Balanceing reducer skew in Map Reduce workloads using progressive smapling. In: Proceedings of the 3rd ACM Symposium on Cloud Computing. 2012, 1–13
Kwon Y C, Balazinska M, Howe B, Rolia J. Skew Tune: mitigating skew in Map Reduce applications. In: Proceedings of ACM SIGMOD International Conference on Management of Data. 2012, 25–36
Chen Q, Liu C, Xiao Z. Improving Map Reduce performance using smart speculative execution strategy. IEEE Transactions on Computers, 2014, 63(4): 954–967
Ananthanarayanan G, Agarwal S, Kandula S, Greenberg A, Stoica I, Harlan D, Harris E. Scarlett: coping with skewed content popularity in Map Reduce clusters. In: Proceedings of the 6th ACM European Conference on Computer Systems. 2011, 287–300
Le Y F, Liu J C, Ergün F. Wang D. Online load balancing for Map Reduce with skewed data input. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2014, 2004–2012
Tang J H, Tay W P, Wen Y G. Dynamic request redirection and elastic service scaling in cloud-centric media networks. IEEE Transactions on Multimedia, 2014, 16(5): 1434–1445
Zhao X M, Ma H D, Zhang H T, Tang Y, Fu G P. Metadata extraction and correction for large-scale traffic surveillance videos. In: Proceedings of IEEE International Conference on Big Data. 2014, 412–420
Neely M J. Stochastic network optimization with application to communication and queueing system. Synthesis Lectures on Communication Networks, 2010 3(1): 1–211
Feris R S, Siddiquie B, Petterson J, Zhai Y, Datta A, Brown L M, Pankanti S. Large-scale vehicle detection, indexing, and search in urban surveillance videos. IEEE Transactions on Multimedia, 2012, 14(1): 28–42
Dean J, Ghemawat S. Map Reduce: simplified data processing on large clusters. In: Proceedings of the USENIX Symposium on Operating Systems Design and Implementation. 2004, 137–150
Zaharia M, Das T, Li H Y, Hunter T, Shenker S, Stoica I. Discretized streams: fault-tolerant streaming computation at scale. In: Proceedings of the 23th ACM Symposium on Operating Systems Principles. 2013, 423–438
Maguluri S T, Strikant R. Scheduling jobs with unkonwn duration in clouds. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2013, 1935–1943
Huang F, Anandkumar A. FCD: fast-concurrent-distributed load balanceing under switch costs and imperfect observation. In: Proceedings of the 33rd Annual IEEE International Conference on Computer Communications. 2013, 1944–1952
Khayyat Z, Awara K, Alonazi A, Jamjoom H, Williams D, Kalnis P. Mizan: a system for dynamic load balancing in large-scale graph processing. In: Proceedings of the 8th ACM European Conference on Computer Systems. 2013, 169–182
Acknowledgements
The research reported in this paper was supported by the National High-Tech R&D Program (863 Program) (2015AA01A705), the NSFC-Guangdong Joint Found (U1501254), the Cosponsored Project of Beijing Committee of Education, and the Beijing Training Project for the Leading Talents in S&T (ljrc 201502).
Author information
Authors and Affiliations
Corresponding author
Additional information
Yihong Gao is now a PhD candidate of School of Computer Science, Beijing University of Posts and Telecommunications, China. His current research mainly focuses on resource scheduling approach, video data center, and cloud computing.
Huadong Ma is a Chang Jiang Scholar professor and director of Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, and also the executive dean of School of Computer Science, Beijing University of Posts and Telecommunications, China. He received his PhD degree in computer science from Institute of Computing Technology, Chinese Academy of Sciences, China in 1995. He was an awardee of the National Science Funds for Distinguished Young Scholars in 2009. His current research focuses on multimedia system and networking, sensor networks and Internet of things, and he has published over 180 papers and four books on these fields. He is a member of IEEE and ACM.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Gao, Y., Ma, H. StreamTune: dynamic resource scheduling approach for workload skew in video data center. Front. Comput. Sci. 12, 669–681 (2018). https://doi.org/10.1007/s11704-016-5438-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-016-5438-1