Skip to main content
SpringerLink
Log in

Text query based summarized event searching interface system using deep learning over cloud

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the digital era, the growth of multimedia data is increasing at a rapid pace, which demands both effective and efficient summarization techniques. Such advanced techniques are required so that the users can quickly access the video content, recorded by multiple cameras for a certain period. At present, it is very challenging to manage and search a huge amount of multiview video data, which contains the inter-views dependencies, significant illumination changes, and many low-active frames. This work highlights an efficient summarization technique to summarize and then search the events in such multi-view videos over cloud through text query. Deep learning framework is employed to extract the features of moving objects in the frames. The inter-views dependencies among multiple views of the video are captured via local alignment. Parallel Virtual Machines (VMs) in the Cloud environment have been used to process the multiple video clip independently at a time. Object tracking is applied to filter the low-active frames. Experimental Results indicate that the model successfully reduces the video content, while preserving the momentous information in the form of the events. A computing analysis also indicates that it meets the requirement of real-time applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://blast.ncbi.nlm.nih.gov/Blast.cgi

  2. https://docs.openstack.org/newton/install/

References

  1. Almeida J, Leite NJ, Torres RDS (2013) Online video summarization on compressed domain. J Vis Commun Image Represent 24(6):729–738

    Article  Google Scholar 

  2. Ayguade E, Navarro JJ, Jimenez-Gonzalez D (2007) Smith-waterman algorithm, daring. Tersedia pada. [Online] available: http://docenciatextit.ac.upc.edu/master/AMPP/slides/ampp_sw_presentation.pdf

  3. Chen K-W, Lai C-C, Lee P-J, Chen C-S, Hung Y-P (2011) Adaptive learning for target tracking and true linking discovering across multiple non-overlapping cameras. IEEE Trans Multimedia 13(4):625–638

    Article  Google Scholar 

  4. Dumont E, Merialdo B (2009) Rushes video parsing using video sequence alignment. In: 2009 seventh international workshop on content-based multimedia indexing. IEEE, pp 44–49

  5. Fu Y, Guo Y, Zhu Y, Liu F, Song C, Zhou Z-H (2010) Multi-view video summarization. IEEE Trans Multimed 12(7):717–729

    Article  Google Scholar 

  6. Gygli M, Grabner H, Gool LV (2015) Video summarization by learning submodular mixtures of objectives. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3090–3098

  7. Hao Y, Mu T, Goulermas JY, Jiang J, Hong R, Wang M (2017) Unsupervised t-distributed video hashing and its deep hashing extension. IEEE Trans Image Process 26(11):5531–5544

    Article  MathSciNet  Google Scholar 

  8. Hong R, Li L, Cai J, Tao D, Wang M, Tian Q (2017) Coherent semantic-visual indexing for large-scale image retrieval in the cloud. IEEE Trans Image Process 26(9):4128–4138

    Article  MathSciNet  Google Scholar 

  9. Jian M, Zhang W, Yu H, Cui C, Nie X, Zhang H, Yin Y (2018) Saliency detection based on directional patches extraction and principal local color contrast. J Vis Commun Image Represent 57:1–11

    Article  Google Scholar 

  10. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  11. Kuanar SK, Ranga KB, Chowdhury AS (2015) Multi-view video summarization using bipartite matching constrained optimum-path forest clustering. IEEE Trans Multimed 17(8):1166–1173

    Article  Google Scholar 

  12. Kumar K, Kurhekar M (2016) Economically efficient virtualization over cloud using docker containers. In: 2016 IEEE international conference on cloud computing in emerging markets (CCEM). IEEE, pp 95–100

  13. Kumar K, Kurhekar M (2017) Sentimentalizer: Docker container utility over Cloud. In: 2017 ninth international conference on advances in pattern recognition (ICAPR). IEEE, pp 1–6

  14. Kumar K, Shrimankar (2017) F-DES: Fast and deep event summarization. IEEE Trans Multimed 20(2):323–334

    Article  Google Scholar 

  15. Kumar K, Shrimankar DD (2018) Deep event learning boost-up approach: Delta. Multimed Tools Appl 77(20):26635–26655

    Article  Google Scholar 

  16. Kumar K, Shrimankar DD (2018) ESUMM: Event summarization on scale-free networks. IETE Technical Review

  17. Kumar K, Shrimankar DD, Singh N (2016) Equal partition based clustering approach for event summarization in videos. In: 2016 12th international conference on signal-image technology & internet-based systems (SITIS). IEEE, pp 119–126

  18. Kumar K, Shrimankar DD, Singh N (2017) Event bagging: A novel event summarization approach in multiview surveillance videos. In: 2017 international conference on innovations in electronics, signal processing and communication (IESC). IEEE, pp 106–111

  19. Kumar K, Shrimankar DD, Singh N (2018) Eratosthenes sieve based key-frame extraction technique for event summarization in videos. Multimedia Tools Appl 77(6):7383–7404

    Article  Google Scholar 

  20. Kumar K, Shrimankar DD, Singh N (2018) SOMES: an efficient SOM technique for event summarization in multi-view surveillance videos. In: Recent findings in intelligent computing techniques. Springer, Singapore, pp 383–389

  21. Kurzhals K, John M, Heimerl F, Kuznecov P, Weiskopf D (2016) Visual movie analytics. IEEE Trans Multimed 18(11):2149–2160

    Article  Google Scholar 

  22. Liu J, Gong M, Qin AK, Tan KC (2019) Bipartite differential neural network for unsupervised image change detection. IEEE Trans Neural Netw Learn Sys 31(3):876–890

    Article  MathSciNet  Google Scholar 

  23. Michael G, John D (1991) Sequence analysis primer. Palgrave Macmillan, London, pp 169–175

    Google Scholar 

  24. Ou S-H, Lee C-H, Somayazulu VS, Chen Y-K, Chien S-Y (2014) On-line multi-view video summarization for wireless video sensor network. IEEE J Select Topics Sig Process 9(1):165–179

    Google Scholar 

  25. Panda R, Roy-Chowdhury AK (2017) Multi-view surveillance video summarization via joint embedding and sparse optimization. IEEE Trans Multimed 19 (9):2010–2021

    Article  Google Scholar 

  26. Panda R, Mithun NC, Roy-Chowdhury AK (2017) Diversity-aware multi-video summarization. IEEE Trans Image Process 26(10):4712–4724

    Article  MathSciNet  Google Scholar 

  27. Potapov D, Douze M, Harchaoui Z, Schmid C (2014) Category-specific video summarization. In: European conference on computer vision. Springer, Cham, pp 540–555

  28. Rav-Acha A, Pritch Y, Peleg S (2006) Making a long video short: Dynamic video synopsis. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol 1. IEEE, pp 435–441

  29. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  30. Taherkhani A, Belatreche A, Li Y, Maguire LP (2018) A supervised learning algorithm for learning precise timing of multiple spikes in multilayer spiking neural networks. IEEE Trans Neural Netw Learn Sys 29(11):5394–5407

    Article  MathSciNet  Google Scholar 

  31. Tong W, Zhong S (2016) A unified resource allocation framework for defending against pollution attacks in wireless network coding systems. IEEE Trans Inform Foren Sec 11(10):2255–2267

    Article  Google Scholar 

  32. Wang C, Zhang H, Yang L, Cao X, Xiong H (2017) Multiple semantic matching on augmented N-partite graph for object co-segmentation. IEEE Trans Image Process 26 (12):5825–5839

    Article  MathSciNet  Google Scholar 

  33. Wong SC, Stamatescu V, Gatt A, Kearney D, Lee I, McDonnell MD (2017) Track everything: Limiting prior knowledge in online multi-object recognition. IEEE Trans Image Process 26(10):4669–4683

    Article  MathSciNet  Google Scholar 

  34. Xu B, Wang X, Jiang Y-G (2016) Fast summarization of user-generated videos: Exploiting semantic, emotional, and quality clues. IEEE MultiMedia 23 (3):23–33

    Article  Google Scholar 

  35. Zhao B, Xing EP (2014) Quasi real-time summarization for consumer videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2513–2520

  36. Zhao Z-Q, Zheng P, Xu S-T, Wu X (2019) Object detection with deep learning: A review. IEEE Trans Neural Netw Learn Sys 30(11):3212–3232

    Article  Google Scholar 

  37. [Online] https://www.ctu.edu.vn/dvxe/Bioinformatic%20course/manuals/blast/blastmanual/fasta.htm. Retrieved (2016)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, National Institute of Technology, Uttarakhand, India

    Krishan Kumar

Authors
  1. Krishan Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Krishan Kumar.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, K. Text query based summarized event searching interface system using deep learning over cloud. Multimed Tools Appl 80, 11079–11094 (2021). https://doi.org/10.1007/s11042-020-10157-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10157-4

Keywords

Search

Navigation