Skip to main content
SpringerLink
Log in

A video cloud platform combing online and offline cloud computing technologies

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

With the rapid growth of video data from various sources, like security and transportation surveillance, there arise requirements for both online real-time analysis and offline batch processing of large-scale video data. Existing video processing systems fall short in addressing many challenges in large-scale video processing, for example performance, data storage, and fault tolerance. The emerging cloud computing and big data techniques shed lights to intelligent processing for large-scale video data. This paper proposes a general cloud-based architecture and platform that can provide a robust solution to intelligent analysis and storage for video data, which is named as BiF (Batch processing Integrated with Fast processing) architecture. We have implemented the BiF architecture using both Hadoop platform and Storm platform, which are typical offline batch processing cloud platform and online real-time processing cloud platform, respectively. The proposed architecture can handle continual surveillance video data effectively, where real-time analysis, batch processing, distributed storage and cloud services are seamlessly integrated to meet the requirements of video data processing and management. The evaluations show that the proposed approach is efficient in terms of performance, storage, and fault tolerance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://storm.apache.org/.

  2. http://kafka.apache.org/.

  3. https://storm.apache.org/documentation/Distributed-RPC.html.

  4. http://sploutsql.com/.

  5. http://webcam-capture.sarxos.pl/.

  6. http://storm.apache.org/documentation/Trident-API-Overview.html.

  7. http://www.nimbusproject.org/.

References

  1. Benfold B, Reid I (2011) Stable multi-target tracking in real-time surveillance video. In: Computer vision and pattern recognition (CVPR), 2011 IEEE conference on IEEE, pp 3457–3464

  2. Chowdhury A, Tripathy SS (2014) Detection of human presence in a surveillance video using fuzzy approach. In: Signal processing and integrated networks (SPIN), 2014 international conference on, IEEE pp 216–219

  3. Zhou H et al (2010) Feature extraction and clustering for dynamic video summarisation. Neurocomputing 73(10):1718–1729

    Article  Google Scholar 

  4. Ali SF, Jaffar J, Malik AS (2011) Proposed framework of intelligent video automatic target recognition system (ivatrs). In: National postgraduate conference (NPC), IEEE 2011, pp. 1–5

  5. Cavallaro A, Steiger O, Ebrahimi T (2005) Tracking video objects in cluttered background. Circuits Syst Video Technol IEEE Trans 15(4):575–584

    Article  Google Scholar 

  6. Miller M (2008) Cloud computing: web-based applications that change the way you work and collaborate online. Que publishing, New York

    Google Scholar 

  7. Neal D, Rahman SM (2012) Video surveillance in the cloud-computing? In: Electrical and computer engineering (ICECE), 2012 7th international conference on, IEEE pp 58–61

  8. Ryu C, Lee D, Jang M, Kim C, Seo E (2013) Extensible video processing framework in apache hadoop. In: Cloud computing technology and science (CloudCom), 2013 IEEE 5th international conference on, IEEE 2, pp 305–310

  9. Zhang W, Duan P, Lu Q (2014) A realtime framework for video object detection with storm. In: The 2014 international symposium on ubicom frontiers - innovative research, systems and technologies (UFirst 2014), IEEE pp 732–737

  10. Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. Commun ACM 51(1):107–113

    Article  Google Scholar 

  11. Shirahama K (2011) Intelligent video processing using data mining techniques. ACM SIGMultimed Rec 3(2):7–9

    Article  Google Scholar 

  12. Chen Z, Ellis T (2014) A self-adaptive gaussian mixture model. Comput Vis Image Underst 122:35–46

    Article  Google Scholar 

  13. Marz N, Warren J (2013) Big data: principles and best practices of scalable realtime data systems. O’Reilly Media, Sebastopol

    Google Scholar 

  14. Kim M, Cui Y, Han S, Lee H (2013) Towards efficient design and implementation of a hadoop-based distributed video transcoding system in cloud computing environment. Int J Multimed Ubiquitous Eng 8(2):213–224

    Google Scholar 

  15. Vora MN (2011) Hadoop-hbase for large-scale data. In: Computer science and network technology (ICCSNT), 2011 international conference on IEEE, vol 1, pp 601–605

  16. Zhu X, Wu X, Elmagarmid AK, Feng Z, Wu L (2005) Video data mining: semantic indexing and event detection from the association perspective. Knowl Data Eng IEEE Trans 17(5):665–677

    Article  Google Scholar 

  17. Shirahama Kimiaki, Iwamoto Kazuhisa, Uehera Kuniaki (2004) Video data mining: rhythms in a movie. In Multimedia and Expo, 2004. ICME’04. 2004 IEEE International Conference on, volume 2, pages 1463–1466. IEEE

  18. Oh JH, Lee JK, Kote S, Bandi B (2003) Multimedia data mining framework for raw video sequences. In: Mining multimedia and complex data, Springer: Berlin pp 18–35

  19. Khan BUI, Olanrewaju RF, Altaf H, Shah A (2014) Critical insight for mapreduce optimization in hadoop. Int J Comput Sci Control Eng 2(1):1–7

    Google Scholar 

  20. Fielding RT, Taylor RN (2002) Principled design of the modern web architecture. ACM Trans Internet Technol 2(2):115–150

    Article  Google Scholar 

  21. Bass L, Clements P, Kazman R (2012) Software architecture in practice. Addison-Wesley, Boston

    Google Scholar 

  22. Zhang W (2014) Klaus Marius Hansen, and Mads Ingstrup. A hybrid approach to self-management in a pervasive service middleware. Knowl-Based Syst 67:143–161

    Article  Google Scholar 

  23. Zhang W, Wang W, Duan P, Liu X, Lu Q (2014) Online multiperson tracking and counting with cloud computing. In: 2014 International conference on identification, information and knowledge in the internet of things, IEEE pp 72–75

  24. Ghemawat S, Gobioff H, Leung S-T (2003) The google file system. In: ACM SIGOPS operating systems review, 37, pp 29–43

  25. White T (2012) Hadoop: the definitive guide. O’Reilly Media, Inc., Sebastopol

    Google Scholar 

  26. Zhang K, Chen X (2014) Large-scale deep belief nets with mapreduce. Access, IEEE

Download references

Acknowledgments

The research in this paper is jointly supported by the National Natural Science Foundation of China (Grant No. 61402533) and Natural Science Foundation of Shandong Province (Grant No. ZR2014FM038), “Key Technologies Development Plan of Qingdao Technical Economic Development Area” and “Start up Funds of Top Professors in China University of Petroleum”.

Author information

Authors and Affiliations

  1. Department of Software Engineering, China University of Petroleum, No.66 Changjiang West Road, Qingdao, 266580, China

    Weishan Zhang, Liang Xu, Pengcheng Duan, Wenjuan Gong & Qinghua Lu

  2. Department of Computer Science and Engineering, Fudan University, Shanghai, China

    Su Yang

Authors
  1. Weishan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pengcheng Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenjuan Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qinghua Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Su Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weishan Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Xu, L., Duan, P. et al. A video cloud platform combing online and offline cloud computing technologies. Pers Ubiquit Comput 19, 1099–1110 (2015). https://doi.org/10.1007/s00779-015-0879-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-015-0879-3

Keywords

Search

Navigation