Skip to main content
SpringerLink
Log in

Real-time institution video data analysis using fog computing and adaptive background subtraction

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The increasing demand for video surveillance systems has led to a surge in research towards developing smart video surveillance systems that can combat the growing levels of insecurity. However, the massive amount of video data generated by these systems has overwhelmed the storage and processing capabilities of analytic applications. This paper proposes a fog computing-based smart video surveillance system that provides less latency, network bandwidth, and response time by localizing data to the edges of the network. The proposed system incorporates two key preprocessing steps, namely adaptive key frame extraction and adaptive contour-based background subtraction, to increase the quality of detecting abnormal motions from surveillance video streams. The adaptive key frame extraction mechanism extracts key frames from the available frames in the video sequence using a sliding window technique. The high-level semantic information of video frames is learned using the visual geometry group-16 Transfer Learning (VGG-16 TL) technique to better represent the video content. The adaptive contour-based background subtraction mechanism separates target foreground pixels from the background scenes, paving the way for easy detection of abnormal motions in the video frames. The proposed system leverages fog computing to process and store data, reducing latency and improving overall performance. The fog layer performs adaptive background subtraction, contour detection, and object analysis, ensuring timely processing of video frames and minimizing the need for transmitting large amounts of data to the cloud. The proposed system is evaluated using a real-time video dataset in terms of accuracy, compression ratio, precision, recall, processing time, latency, and network bandwidth, demonstrating the efficacy of the proposed system for abnormal motion detection and data transmission. Overall, the proposed fog computing-based smart video surveillance system provides an effective solution for detecting abnormal motions in real-time institution video data with reduced latency, network bandwidth, and response time, demonstrating the potential of fog computing for video surveillance applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Availability of data and materials

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Patrikar, D.R., Parate, M.R.: Anomaly detection using edge computing in video surveillance system. Int. J. Multimedia Inform. Retriev. 11(2), 85–110 (2022)

    Article  Google Scholar 

  2. Kong, M., Guo, Y., Alkhazragi, O., Sait, M., Kang, C.H., Ng, T.K., Ooi, B.S.: Real-time optical-wireless video surveillance system for high visual-fidelity underwater monitoring. IEEE Photon. J. 14(2), 1–9 (2022)

    Article  Google Scholar 

  3. Roa’a, M., Aljazaery, I.A., ALRikabi, H.T.S., Alaidi, A.H.M.: Automated cheating detection based on video surveillance in the examination classes. iJIM, 16(08), p.125 (2022)

  4. Tseng, C.H., Hsieh, C.C., Jwo, D.J., Wu, J.H., Sheu, R.K., Chen, L.C.: Person retrieval in video surveillance using deep learning–based instance segmentation. J. Sens. 2021, 1–12 (2021)

    Article  Google Scholar 

  5. Dong, Y., Zhang, Y., Zhang, J., Zhang, X.: Video key frame extraction based on scale and direction analysis. J. Eng. 9, 910–918 (2022)

    Article  Google Scholar 

  6. Surya Priya, M., Diana Josephine, D., Abinaya, P.: IOT based smart and secure surveillance system using video summarization. In advances in computing and network communications: Proceedings of CoCoNet 2020, Volume 1 (pp. 423–435). Springer Singapore (2021)

  7. Pervaiz, M., Jalal, A., Kim, K.: Hybrid algorithm for multi people counting and tracking for smart surveillance. In 2021 International Bhurban conference on applied sciences and technologies (IBCAST) (pp. 530–535). IEEE (2021)

  8. Murugesan, M., Thilagamani, S.: Efficient anomaly detection in surveillance videos based on multi layer perception recurrent neural network. Microproc. Microsyst. 79, 103303 (2020)

    Article  Google Scholar 

  9. Fathy, C., Saleh, S.N.: Integrating deep learning-based iot and fog computing with software-defined networking for detecting weapons in video surveillance systems. Sensors 22(14), 5075 (2022)

    Article  Google Scholar 

  10. Rawat, R., Chakrawarti, R.K., Vyas, P., Gonzáles, J.L.A., Sikarwar, R., Bhardwaj, R.: Intelligent Fog computing surveillance system for crime and vulnerability identification and tracing. Int. J. Inform. Secur. Privacy (IJISP) 17(1), 1–25 (2023)

    Article  Google Scholar 

  11. Elahi, G.M.E., Yang, Y.H.: Online learnable keyframe extraction in videos and its application with semantic word vector in action recognition. Pattern Recogn. 122, 108273 (2022)

    Article  Google Scholar 

  12. Nandini, H.M., Chethan, H.K., Rashmi, B.S.: Shot based keyframe extraction using edge-LBP approach. J. King Saud University Comput. Inform. Sci. 34(7), 4537–4545 (2022)

    Google Scholar 

  13. Wang, Z., Zhu, Y.: Video key frame monitoring algorithm and virtual reality display based on motion vector. IEEE Access 8, 159027–159038 (2020)

    Article  Google Scholar 

  14. Kumar, B.S., Seetharaman, K.: Video sequence feature extraction and segmentation using likelihood regression model. Multimedia Tools Applic. 80, 24343–24361 (2021)

    Article  Google Scholar 

  15. Giraldo, J.H., Bouwmans, T.: GraphBGS: Background subtraction via recovery of graph signals. In 2020 25th International Conference on Pattern Recognition (ICPR) (pp. 6881–6888). IEEE (2021)

  16. Elhoseny, M.: Multi-object detection and tracking (MODT) machine learning model for real-time video surveillance systems. Circuits Syst. Signal Process. 39, 611–630 (2020)

    Article  Google Scholar 

  17. Tezcan, M.O., Ishwar, P., Konrad, J.: BSUV-Net 2.0: Spatio-temporal data augmentations for video-agnostic supervised background subtraction. IEEE Access, 9, 53849–53860. (2021).

  18. Zhao, C., Hu, K., Basu, A.: Universal background subtraction based on arithmetic distribution neural network. IEEE Trans. Image Process. 31, 2934–2949 (2022)

    Article  Google Scholar 

  19. Kamate, S., Yilmazer, N.: Application of object detection and tracking techniques for unmanned aerial vehicles. Procedia Computer Sci. 61, 436–441 (2015)

    Article  Google Scholar 

  20. Maddalena, L., Petrosino, A.: A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans. Image Process. 17(7), 1168–1177 (2008)

    Article  MathSciNet  Google Scholar 

  21. Zhao, Z., Zou, W., Wang, J.: Action recognition based on C3D network and adaptive keyframe extraction. In 2020 IEEE 6th International Conference on Computer and Communications (ICCC) (pp. 2441–2447). IEEE (2020)

  22. Dibs, H., Hasab, H.A., Jaber, H.S., Al-Ansari, N.: Automatic feature extraction and matching modelling for highly noise near-equatorial satellite images. Innov. Infrastr. Solut. 7(1), 2 (2022)

    Article  Google Scholar 

  23. Kapre, B.S., Rajurkarb, A.M.: Key-frame extraction based video watermarking using speeded up robust features and discrete cosine transform. Comput. Sci. Inform. Technol. 4, 85–94 (2023)

    Article  Google Scholar 

  24. Jose, J.T., Rajkumar, S., Ghalib, M.R., Shankar, A., Sharma, P., Khosravi, M.R.: Efficient Shot Boundary Detection with Multiple Visual Representations. Mobile Inform. Syst (2022)

  25. Liu, Q., Yang, S., Liu, J., Xiong, P., Zhou, M.: A discrete wavelet transform and singular value decomposition-based digital video watermark method. Appl. Math. Model. 85, 273–293 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  26. Ponni alias Sathya, S., Ramakrishnan, S.: Non‐redundant frame identification and keyframe selection in DWT‐PCA domain for authentication of video. IET Image Process. 14(2), 366–375 (2020)

  27. Gharbi, H., Bahroun, S., Zagrouba, E.: Key frame extraction for video summarization using local description and repeatability graph clustering. SIViP 13, 507–515 (2019)

    Article  Google Scholar 

  28. Tang, H., Ding, L., Wu, S., Ren, B., Sebe, N., Rota, P.: Deep unsupervised key frame extraction for efficient video classification. ACM Trans. Multimed. Comput. Commun. Appl. 19(3), 1–17 (2023)

    Article  Google Scholar 

  29. Meghana, R.K., Chitkara, Y., Apoorva, S.: Background-modelling techniques for foreground detection and Tracking using Gaussian Mixture Model. In 2019 3rd international conference on computing methodologies and communication (ICCMC) (pp. 1129–1134). IEEE (2019)

  30. Agrawal, S., Natu, P.: An improved Gaussian Mixture Method based background subtraction model for moving object detection in outdoor scene. In 2021 Fourth International Conference on Electrical, Computer and Communication Technologies (ICECCT) (pp. 1–8). IEEE (2021)

  31. Deffo, L.L.S., Tagne, E.F.: INBaSA: Novel Non-parametric background subtraction algorithm based on and improved pixel based segmentation. In CARI (2022)

  32. Li, S., Wu, J., Long, C., Lin, Y.B.: A full-process optimization-based background subtraction for moving object detection on general-purpose embedded devices. IEEE Trans. Consum. Electron. 67(2), 129–140 (2021)

    Article  Google Scholar 

  33. Cokbas, M., Ishwar, P., Konrad, J.: Low-resolution overhead thermal tripwire for occupancy estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 88–89) (2020)

  34. Fischer, J., Doolan, C.: An improved eigenvalue background noise reduction method for acoustic beamforming. Mech. Syst. Signal Process. 140, 106702 (2020)

    Article  Google Scholar 

  35. Panda, S., Nanda, P.K.: Kernel density estimation and correntropy based background modeling and camera model parameter estimation for underwater video object detection. Soft. Comput. 25(15), 10477–10496 (2021)

    Article  Google Scholar 

  36. Acharya, S., Nanda, P.K.: Adjacent LBP and LTP based background modeling with mixed-mode learning for foreground detection. Pattern Anal. Appl. 24, 1047–1074 (2021)

    Article  Google Scholar 

  37. Tezcan, O., Ishwar, P., Konrad, J.: BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 2774–2783) (2020)

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Computing Technologies, SRM Institute of Science and Technology, Kattankulathur, Chennai, 603203, India

    R. S. Amshavalli & J. Kalaivani

Authors
  1. R. S. Amshavalli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Kalaivani
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ARS agreed on the content of the study. ARS and JK collected all the data for analysis. ARS agreed on the methodology. ARS and JK completed the analysis based on agreed steps. Results and conclusions are discussed and written together. All authors read and approved the final manuscript.

Corresponding author

Correspondence to R. S. Amshavalli.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amshavalli, R.S., Kalaivani, J. Real-time institution video data analysis using fog computing and adaptive background subtraction. J Real-Time Image Proc 20, 96 (2023). https://doi.org/10.1007/s11554-023-01350-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01350-3

Keywords

Search

Navigation